dd26d35137
* learnings: surface preamble in TOC, mirror line-boundary truncation server-side, fix empty-repo intro copy three audit fixes on top of the recent learnings overhaul (#717): - `parseLearningsHeadings` now prepends a synthetic `(preamble)` entry when a body has non-whitespace content before the first heading. the prompt instructs the agent NOT to slurp the whole file when a TOC is present, so without this any preamble lines were silently invisible (realistic transitional case: an agent partially restructures a legacy free-text body and leaves bullets above the first `## `). - server-side PATCH route now applies the same line-boundary-aware truncation as the action (defense in depth via a shared `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` exported from `action/internal`). the raw `.slice` it used before could leave a mid-heading tail on any caller that bypassed the client-side truncate, breaking the next-seed TOC parse. removes the duplicated cap constant. - `buildLearningsSection` intro no longer asserts "accumulated by previous agent runs" — false for fresh repos with zero history. new copy is tense-neutral and works for empty + populated bodies. also nudges the agent to re-read after mid-run edits (the inlined TOC ranges are a run-start snapshot). Co-authored-by: Cursor <cursoragent@cursor.com> * learnings prompt: tighten to single evergreen test, allow tool-quirk bullets when they prevent repeat waste The blanket "no pullfrog tool quirks" ban was wrong — if the agent burned calls discovering a quirk this run, recording the workaround prevents the next run from repeating the waste. Reframe around one litmus ("would a future run do its work better because this bullet exists?") and trust it to subsume the scattered don'ts. Drop the 3+ months timeframe (arbitrary) and the four-example pullfrog/PR/date/play-by-play list (the rule underneath is "don't anchor facts to repo state that will move"). Cuts ~10 lines from a prompt the model was already mostly ignoring; the remaining anchor list is narrower and more enforceable. * audit-learnings-r2: align wiki + tighten re-read nudge - wiki/prompt.md described the post-run reflection prompt as "bans pullfrog-tool quirks (those belong in tool descriptions, not per-repo learnings), bans PR/review/commit/date references" — that's stale after the prompt rewrite. update to: single-litmus framing, expanded anchor list (now includes version pins + line numbers), and explicit allowance for tool-quirk workarounds when discovery burned calls. - buildLearningsSection re-read nudge said "re-read after editing" which can be read as "re-read the section you edited". in fact any edit shifts the line numbers of every later section in the TOC, not just the edited one. tighten to make that explicit. mirror the new wording in the wiki example block. update the test substring assertion accordingly. * postRun: refresh JSDoc to match the reflection prompt rewrite `buildLearningsReflectionPrompt`'s JSDoc still listed "PR-/review-/commit-/date-anchored facts" and "rediscovery of pullfrog-tool quirks" as failure modes the prompt pushes back on. after b586b4f8 the prompt no longer bans tool-quirk bullets (it explicitly allows them when the agent burned calls discovering the quirk), and the anchor list expanded to cover branch refs, version pins, and line numbers too. update the JSDoc so it describes the prompt that actually exists, and call out the cross-repo drift tradeoff that comes with allowing tool-quirk bullets. * fix(mcp/issueEvents): narrow event.event before Set.has lookup octokit's listEventsForTimeline union includes timeline-event members where `event` is `event?: string`. `("event" in event)` does not narrow that property to non-undefined, so `relevantEventTypes.has(event.event)` was passing `string | undefined` to a `Set<string>.has`. typescript only flagged this once `cf-worker-indexing` started seeing the file via the type graph that now reaches mcp through the new `truncateAtLineBoundary` re-export in `action/internal/index.ts`. fix the latent bug at the source: require `typeof event.event === "string"` before the Set lookup. * learnings: split truncation helpers into MCP-free module re-exporting `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` from `action/utils/learnings.ts` through `action/internal/index.ts` accidentally pulled the entire MCP type graph into the SDK barrel: `learnings.ts` imports `ToolContext` from `mcp/server.ts`, which transitively wires every tool module under `action/mcp/` into anything that imports from `pullfrog/internal`. for `cf-worker-indexing/tsconfig.json` (`customConditions: ["@pullfrog/source"]`) and the root `tsc` (which compiles the proprietary app routes that import from `pullfrog/internal`), this expanded the type-checked surface and surfaced two latent issues in unrelated files (`mcp/issueEvents.ts`, `utils/subprocess.ts`). a 6-line pure string helper has no business dragging mcp/server.ts into anyone else's type graph. move both symbols to `action/utils/learningsTruncate.ts`. `learnings.ts` re-exports them so existing callers keep working; `internal/index.ts` re-exports from the truncate-only module so the SDK barrel stays MCP-free. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
117 lines
4.2 KiB
TypeScript
117 lines
4.2 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import { buildLearningsSection, renderLearningsToc } from "./instructions.ts";
|
|
import type { LearningsHeading } from "./runContext.ts";
|
|
|
|
const h = (depth: 1 | 2 | 3 | 4 | 5 | 6, title: string, startLine: number, endLine: number) => ({
|
|
depth,
|
|
title,
|
|
startLine,
|
|
endLine,
|
|
});
|
|
|
|
describe("renderLearningsToc", () => {
|
|
it("renders flat h2 list with parenthesized ranges, no hashes or backticks", () => {
|
|
const headings: LearningsHeading[] = [
|
|
h(2, "Build & test", 1, 18),
|
|
h(2, "Architecture", 19, 60),
|
|
];
|
|
expect(renderLearningsToc(headings)).toBe(
|
|
`- Build & test (L1-L18)
|
|
- Architecture (L19-L60)`
|
|
);
|
|
});
|
|
|
|
it("indents deeper headings 2 spaces per depth level past the shallowest", () => {
|
|
const headings: LearningsHeading[] = [
|
|
h(2, "Build & test", 1, 42),
|
|
h(3, "Local", 3, 18),
|
|
h(3, "CI", 19, 42),
|
|
h(2, "Architecture", 43, 210),
|
|
h(3, "Background workers", 80, 210),
|
|
];
|
|
expect(renderLearningsToc(headings)).toBe(
|
|
`- Build & test (L1-L42)
|
|
- Local (L3-L18)
|
|
- CI (L19-L42)
|
|
- Architecture (L43-L210)
|
|
- Background workers (L80-L210)`
|
|
);
|
|
});
|
|
|
|
it("treats the shallowest depth as the root column when no h2 is present", () => {
|
|
const headings: LearningsHeading[] = [h(3, "Only h3", 1, 5), h(4, "Sub h4", 2, 5)];
|
|
expect(renderLearningsToc(headings)).toBe(
|
|
`- Only h3 (L1-L5)
|
|
- Sub h4 (L2-L5)`
|
|
);
|
|
});
|
|
|
|
it("supports depths up through h6 with stable 2-space indent steps", () => {
|
|
const headings: LearningsHeading[] = [
|
|
h(2, "Two", 1, 10),
|
|
h(3, "Three", 2, 10),
|
|
h(4, "Four", 3, 10),
|
|
h(5, "Five", 4, 10),
|
|
h(6, "Six", 5, 10),
|
|
];
|
|
expect(renderLearningsToc(headings)).toBe(
|
|
`- Two (L1-L10)
|
|
- Three (L2-L10)
|
|
- Four (L3-L10)
|
|
- Five (L4-L10)
|
|
- Six (L5-L10)`
|
|
);
|
|
});
|
|
});
|
|
|
|
describe("buildLearningsSection", () => {
|
|
it("returns empty string when no file path (seed step failed)", () => {
|
|
expect(buildLearningsSection({ filePath: null, headings: [] })).toBe("");
|
|
});
|
|
|
|
it("renders the no-headings affordance when the body has no structure", () => {
|
|
const out = buildLearningsSection({
|
|
filePath: "/tmp/run-1/pullfrog-learnings.md",
|
|
headings: [],
|
|
});
|
|
expect(out).toContain("************* LEARNINGS *************");
|
|
expect(out).toContain("/tmp/run-1/pullfrog-learnings.md");
|
|
expect(out).toContain("no headings yet");
|
|
expect(out).toContain("structure it with");
|
|
// does not include a TOC list when there are no headings
|
|
expect(out).not.toMatch(/\(L\d+-L\d+\)/);
|
|
});
|
|
|
|
it("intro phrasing does not assert prior runs — works for fresh empty repos too", () => {
|
|
const out = buildLearningsSection({
|
|
filePath: "/tmp/run-1/pullfrog-learnings.md",
|
|
headings: [],
|
|
});
|
|
// load-bearing: fresh repos have zero previous runs. the prior copy
|
|
// ("accumulated by previous agent runs") was a lie in that case.
|
|
expect(out).not.toContain("accumulated by previous agent runs");
|
|
expect(out).toContain("maintained across runs");
|
|
});
|
|
|
|
it("renders the TOC inline with the file path and heading guidance", () => {
|
|
const out = buildLearningsSection({
|
|
filePath: "/tmp/run-1/pullfrog-learnings.md",
|
|
headings: [h(2, "Build & test", 1, 18), h(2, "Architecture", 19, 60)],
|
|
});
|
|
expect(out).toContain("************* LEARNINGS *************");
|
|
expect(out).toContain("/tmp/run-1/pullfrog-learnings.md");
|
|
expect(out).toContain("- Build & test (L1-L18)");
|
|
expect(out).toContain("- Architecture (L19-L60)");
|
|
expect(out).toContain("Each range starts at the section heading line");
|
|
// re-read affordance: ranges reflect the run-start snapshot, so the
|
|
// agent needs an explicit nudge to re-read after any mid-run edits.
|
|
// mid-run edits shift the line numbers of every later section, not
|
|
// just the edited one — wording is explicit about that.
|
|
expect(out).toContain("run-start snapshot");
|
|
expect(out).toContain("any edit shifts the line numbers of every later section");
|
|
// explicit "no hashes, no backticks" in the rendered list
|
|
expect(out).not.toContain("- `## Build");
|
|
expect(out).not.toContain("`## Build");
|
|
});
|
|
});
|