Colin McDonnell
d6de1c369a
* learnings: edit-in-place tmpfile (drop update_learnings tool)
learnings now follow the PR-summary file pattern: server seeds
`pullfrog-learnings.md` from `Repo.learnings` at startup, agent reads
it as part of context, may edit in place during the post-run reflection
turn, server reads back at end-of-run and PATCHes if changed.
motivation: `update_learnings` required the agent to pass the FULL
merged list as a string parameter — an output-token tax that grew
linearly with the learnings size, and a constant prompt-context
expansion since the contents were also inlined into the LEARNINGS
section. for repos with mature learnings the prompt was getting
visibly noisy in CI logs.
key changes:
- new `action/utils/learnings.ts` (seed/read helpers + 10k cap)
- `main.ts`: always seed; `persistLearnings` mirrors `persistSummary`
(success path, error path, exit-signal handler, idempotent guard,
byte-trim equality skip); forwards `model` for `LearningsRevision.model`
- `LEARNINGS` prompt section now contains only the file path + a
one-line "read it" instruction (no contents inlined)
- `update_learnings` MCP tool deleted; `action/mcp/learnings.ts` removed
- reflection turn (`buildLearningsReflectionPrompt`) reframed around
file editing with explicit prune-stale + leave-alone-if-nothing-new
framing
- `learningsStep` removed from every mode checklist — surface lives only
in the LEARNINGS prompt section + the reflection turn now
* learnings: harden seed step + refresh stale docs (review feedback)
Three findings from PR review, all implemented:
1. wrap learnings seed in best-effort try/catch (action/main.ts) —
the always-on seed block ran unconditionally and an unwrapped
`seedLearningsFile` (mkdir + writeFile) failure (ENOSPC, EACCES,
hostile sandbox) would unwind into the outer main() catch and flip
an otherwise-successful run to "❌ Pullfrog failed" before the
agent even started. asymmetric with `persistLearnings`'s own
best-effort contract. wrap and log on failure; downstream
consumers (`persistLearnings`, agent harnesses, `resolveInstructions`)
already handle `learningsFilePath: undefined` cleanly.
2. refresh wiki/main.md — `resolveInstructions` parameter renamed
from `learnings` to `learningsFilePath` in this PR; the data-flow
diagram and the resolver dependency table both still showed the
pre-refactor signature.
3. drop deleted `learnings.ts` from ROADMAP.md + RESEARCH.md
"missing MCP tool tests" bullets — `action/mcp/learnings.ts` was
removed in this PR; the bullets are otherwise still accurate.
71 lines
2.8 KiB
TypeScript
71 lines
2.8 KiB
TypeScript
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
|
import {
|
|
LEARNINGS_FILE_NAME,
|
|
learningsFilePath,
|
|
readLearningsFile,
|
|
seedLearningsFile,
|
|
} from "./learnings.ts";
|
|
|
|
describe("learnings tmpfile round-trip", () => {
|
|
let dir: string;
|
|
|
|
beforeEach(async () => {
|
|
dir = await mkdtemp(join(tmpdir(), "pullfrog-learnings-test-"));
|
|
});
|
|
|
|
afterEach(async () => {
|
|
await rm(dir, { recursive: true, force: true });
|
|
});
|
|
|
|
it("seeds with existing learnings and reads them back verbatim", async () => {
|
|
const current = "- run tests with `pnpm -r test`\n- default branch is `main`";
|
|
const path = await seedLearningsFile({ tmpdir: dir, current });
|
|
expect(path).toBe(learningsFilePath(dir));
|
|
expect(path.endsWith(LEARNINGS_FILE_NAME)).toBe(true);
|
|
const read = await readLearningsFile(path);
|
|
expect(read).toBe(current);
|
|
});
|
|
|
|
it("seeds an empty file when the repo has no learnings yet", async () => {
|
|
// empty seed (vs scaffold-with-comment) keeps the byte-trim equality
|
|
// gate clean: an untouched first run reads back as "" and persistLearnings
|
|
// skips the API round-trip rather than writing a placeholder string into
|
|
// Repo.learnings.
|
|
const path = await seedLearningsFile({ tmpdir: dir, current: null });
|
|
const read = await readLearningsFile(path);
|
|
expect(read).toBe("");
|
|
});
|
|
|
|
it("returns null when the file is missing (treated as no-change by persist)", async () => {
|
|
const path = learningsFilePath(dir);
|
|
const read = await readLearningsFile(path);
|
|
expect(read).toBeNull();
|
|
});
|
|
|
|
it("trims whitespace so trailing newlines never trigger a spurious PATCH", async () => {
|
|
// editors commonly add a trailing newline on save. without trimming, a
|
|
// round-trip "read seed → save unchanged" would fail byte-equality and
|
|
// burn a LearningsRevision row on every run.
|
|
const current = "- one fact";
|
|
const path = await seedLearningsFile({ tmpdir: dir, current });
|
|
await writeFile(path, `${current}\n\n `, "utf8");
|
|
const read = await readLearningsFile(path);
|
|
expect(read).toBe(current);
|
|
});
|
|
|
|
it("truncates content over the 10k server-side cap", async () => {
|
|
// server enforces MAX_LEARNINGS_LENGTH = 10_000. truncating client-side
|
|
// avoids a 400 round-trip and keeps the bytes the agent will see in the
|
|
// next run aligned with what the server actually stored.
|
|
const oversized = "x".repeat(11_000);
|
|
const path = await seedLearningsFile({ tmpdir: dir, current: null });
|
|
await writeFile(path, oversized, "utf8");
|
|
const read = await readLearningsFile(path);
|
|
expect(read).toBeTruthy();
|
|
expect(read?.length).toBe(10_000);
|
|
});
|
|
});
|