learnings: TOC + section taxonomy + 100k cap, hygiene rules, tool-quirk descriptions (#717)

* audit learnings: reshape reflection prompt + bake tool quirks into descriptions (#619)

Cross-repo audit of the 48 repos with non-null learnings turned up two
recurring failure modes:

1. ~25-30% of bullets across the most-active repos are pullfrog-tool
   quirks ("shell timeout is in milliseconds", "git args must be a JSON
   array", "create_pull_request_review drops out-of-hunk comments",
   "push_branch may report timeout when push succeeded", "checkout_pr
   shallow.lock retries", "commit_id needs full 40-char SHA"). These are
   universal across repos and should live in tool descriptions, not be
   rediscovered and stored 48 times. Tool descriptions now surface them.

2. Bullets are routinely 200-1000 chars (paragraph-length), and 12 of 48
   repos are at the 10k cap. The reflection prompt now: caps bullets at
   ~240 chars (one specific fact), bans PR/review/commit/date-anchored
   facts that decay within weeks, bans tool-quirk learnings, and tells
   the agent that cap pressure means compress+prune existing bullets,
   not skip new findings.

Co-authored-by: Cursor <cursoragent@cursor.com>

* learnings: add server-generated TOC, fixed section taxonomy, raise cap to 100k (#707)

Cap goes 10k → 100k. Reads stay bounded because the seeded file now
opens with a server-generated table of contents listing every `## `
section's line range — agents read the TOC, then `read_file offset/limit`
just the sections relevant to the current task instead of slurping the
whole file.

## Section taxonomy (fixed)

`## Build & test`, `## CI`, `## Conventions`, `## Architecture`,
`## Gotchas`. Free-form `### ` sub-headings inside a section are fine.
Pre-taxonomy free-text rows get wrapped in a `## Legacy` carve-out on
first seed so they remain visible while the agent gradually re-curates
them during reflection turns.

## Storage shape unchanged

`Repo.learnings` still holds raw markdown (no schema migration). The TOC
is a pure read-side affordance: prepended at seed time, stripped from
the agent-edited file before persist. Markers
`<!-- pullfrog-learnings-toc:* -->` delimit the strip region. Agent
edits inside the markers are discarded.

## Round-trip semantics

`seedLearningsFile` now returns `{ path, canonicalSeed }` where
`canonicalSeed` is the post-TOC body — same shape `readLearningsFile`
returns at end-of-run, so `persistLearnings` byte-compares them
directly to skip the no-op PATCH. Empty-repo first runs end up with the
section scaffold both as seed and as read-back, so untouched runs still
short-circuit cleanly.

## Reflection prompt

Adds explicit section-placement guidance (place each new bullet under
the most relevant `## `; do NOT add new top-level headings; do NOT
edit anything between the TOC markers). Carries forward the bullet
hygiene from the previous commit: ≤240 chars per bullet, no
pullfrog-tool quirks (those belong in tool descriptions), no
PR/review/commit/date references. The "near cap" framing is replaced
with "compress and prune within a section when it grows noisy" since
the cap pressure that drove cramming is gone.

Co-authored-by: Cursor <cursoragent@cursor.com>

* anneal round 1: line-anchored taxonomy detect, partial-merge, line-boundary truncation, scaffold-empty UI

Multi-lens review of the TOC + taxonomy diff surfaced a cluster of
correctness and operational bugs. Fixes:

- `hasAnyTaxonomyHeading` used `String.includes("## X")` which
  false-positives on `### X` (the `## ` substring sits inside `### `),
  prose containing `## CI`, fenced code documenting markdown, etc.
  Replaced with a line-anchored predicate that reuses `parseHeadings`
  so detection and TOC construction stay consistent.

- The "any heading present → pass through verbatim" rule meant a body
  with one taxonomy heading would seed without the other four. Worse,
  requiring all five would flip a body back into Legacy when the agent
  legitimately pruned a section to empty. New `partial` kind: keep
  existing content in place, append missing sections in canonical order
  so the agent always has the full scaffold without losing pruning
  intent.

- `stripLearningsToc` collapsed `\n{3,}` globally; `canonicalSeed`
  doesn't, so an untouched body with intentional triple-newline spacing
  would compare unequal and burn a spurious LearningsRevision row each
  run. Drop the global collapse — only the leading newlines that the
  strip itself introduces are normalized.

- 100k truncation via `slice(0, 100_000)` could cut mid-line, breaking
  `parseHeadings` (whole-line `^## `) on the next seed and flipping a
  cut body back into Legacy. New `truncateAtLineBoundary` cuts at the
  last newline before the cap.

- `LearningsSection.tsx` rendered a scaffold-only body as "has
  learnings" instead of the empty placeholder. Added a
  `hasOnlyEmptyScaffold` guard so the console behaves the same as
  pre-PR for the empty case.

- Seed log line distinguishes `kind=structured/partial/legacy-wrapped/
  empty` instead of `existing=yes/no`, so operators can spot legacy
  migration activity in logs.

- New tests cover: substring false-positive (`### Build & test`,
  in-prose mentions), partial-taxonomy merge (no Legacy wrap),
  full-taxonomy structured pass-through, last-newline truncation,
  triple-newline preservation.

Deferred (documented in PR body): deploy-ordering footgun (action
before API), rollback for rows >10k, Gemini sanitizer dropping
`description` on `anyOf` branches, reflection-on-failed-runs.

Co-authored-by: Cursor <cursoragent@cursor.com>

* anneal r2: hard-truncate fallback when line boundary discards >4k

Round-2 review caught a regression in `truncateAtLineBoundary`: when the
only newline within the first 100k chars sits near the start (e.g. one
heading + 100k+ char single line — pathological pasted log dumps), the
line-boundary cut discards almost all of the body. losing one partial
line is preferable to losing kilobytes; threshold the fallback at 4k.

Co-authored-by: Cursor <cursoragent@cursor.com>

* move TOC out of file: prompt-side rendering, server-parsed headings

drops the in-file TOC + fixed taxonomy in favor of:
- file on disk = verbatim Repo.learnings (no markers, no scaffold)
- server parses headings (mdast-util-from-markdown) at run-context time
  and returns them as RepoSettings.learningsHeadings
- action renders heading TOC into the LEARNINGS prompt section as
  parenthesized line ranges like `Build & test (L1-L42)` with hierarchy
  via 2-space indent off the shallowest depth
- reflection prompt teaches agent-curated structure with a soft 300-line
  per-section cap and explicit guidance to restructure flat legacy lists

cuts 8 helpers (ensureSections, stripLearningsToc, assembleFile,
buildTocBlock, parseHeadings, buildSectionScaffold, hasAnyTaxonomyHeading,
LEARNINGS_SECTIONS) and the canonicalSeed round-trip dance.

action seedLearningsFile is now { path } only; main.ts byte-compares the
trimmed read-back against (current ?? "").trim() to gate the persist
PATCH. truncateAtLineBoundary kept for safety.

new tests:
- test/learningsToc.test.ts (11 parser cases incl. fenced-code, blockquote,
  arbitrary h1-h6 nesting, startLine-points-at-heading invariant)
- action/utils/learningsTocRender.test.ts (7 renderer cases)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>

utils/instructions.ts

@@ -4,6 +4,7 @@ import { encode as toonEncode } from "@toon-format/toon";
 import { type AgentId, formatMcpToolRef, type PayloadEvent, pullfrogMcpName } from "../external.ts";
 import type { Mode } from "../modes.ts";
 import type { ResolvedPayload } from "./payload.ts";
+import type { LearningsHeading } from "./runContext.ts";
 import type { RunContextData } from "./runContextData.ts";
 
 interface InstructionsContext {
@@ -16,6 +17,10 @@ interface InstructionsContext {
    * couldn't be seeded for some reason. main.ts always seeds, so in
    * practice this is always set; the null case keeps the type honest. */
   learningsFilePath: string | null;
+  /** server-parsed TOC for the body of the learnings tmpfile. rendered
+   * inline into the LEARNINGS prompt section so the agent can `read_file`
+   * targeted line ranges instead of pulling the whole file into context. */
+  learningsHeadings: LearningsHeading[];
 }
 
 interface PromptContext extends InstructionsContext {
@@ -369,6 +374,39 @@ export interface ResolvedInstructions {
   runtime: string;
 }
 
+/** render the heading list as an indented bullet TOC. ranges shown in
+ * parentheses (`(L3-L18)`); the start line is always the heading line
+ * itself, so reading the listed range gives the agent the heading +
+ * body together. shallowest heading depth in the body sits at the root
+ * column; deeper levels indent by `(depth - rootDepth) * 2` spaces. */
+export function renderLearningsToc(headings: LearningsHeading[]): string {
+  if (headings.length === 0) return "";
+  const rootDepth = Math.min(...headings.map((h) => h.depth));
+  return headings
+    .map((h) => {
+      const indent = " ".repeat((h.depth - rootDepth) * 2);
+      return `${indent}- ${h.title} (L${h.startLine}-L${h.endLine})`;
+    })
+    .join("\n");
+}
+
+/** assemble the LEARNINGS prompt section: file path + intro + either
+ * the rendered heading TOC (when the body has structure) or a no-headings
+ * affordance pointing the agent at the reflection turn for restructuring.
+ * empty string when the seed step failed and there's no path to surface. */
+export function buildLearningsSection(ctx: {
+  filePath: string | null;
+  headings: LearningsHeading[];
+}): string {
+  if (!ctx.filePath) return "";
+  const intro = `Repo-level learnings accumulated by previous agent runs live at \`${ctx.filePath}\`. Use this file as durable context (test commands, conventions, gotchas, architecture notes).`;
+  const tocBody =
+    ctx.headings.length === 0
+      ? "(no headings yet — file is empty or a flat list. read the whole file. during the post-run reflection turn, structure it with `## ` / `### ` headings so future runs can read targeted ranges.)"
+      : `Read targeted line ranges via your native file tool — do NOT slurp the whole file. Each range starts at the section heading line, so reading the range gives you heading + body together.\n\n${renderLearningsToc(ctx.headings)}`;
+  return `************* LEARNINGS *************\n\n${intro}\n\n${tocBody}`;
+}
+
 function assembleFullPrompt(ctx: {
   toc: string;
   task: string;
@@ -376,17 +414,18 @@ function assembleFullPrompt(ctx: {
   eventContext: string;
   system: string;
   learningsFilePath: string | null;
+  learningsHeadings: LearningsHeading[];
   runtime: string;
 }): string {
-  // the LEARNINGS section is intentionally tiny — just the file path and a
-  // one-line "read it" instruction. embedding the contents would re-inflate
-  // the prompt every run (the previous design's failure mode) and clutter
-  // CI logs. the agent reads the file with its native file tool; the
-  // post-run reflection turn (action/agents/postRun.ts) is where editing
-  // is encouraged, with the prune-stale framing.
-  const learningsSection = ctx.learningsFilePath
-    ? `************* LEARNINGS *************\n\nRepo-level learnings accumulated by previous agent runs live at \`${ctx.learningsFilePath}\`. Read this file early and let the entries inform your approach (test commands, conventions, gotchas, etc.). The file may be empty if no learnings have been collected yet.`
-    : "";
+  // server-parsed TOC is rendered inline so the agent can target line
+  // ranges via its native file tool. the file body itself is never
+  // inlined — that would re-inflate context every run and clutter CI
+  // logs. post-run reflection (action/agents/postRun.ts) is where
+  // editing is encouraged.
+  const learningsSection = buildLearningsSection({
+    filePath: ctx.learningsFilePath,
+    headings: ctx.learningsHeadings,
+  });
 
   const runtimeSection = `************* RUNTIME *************\n\n${ctx.runtime}`;
 
@@ -421,7 +460,10 @@ export function resolveInstructions(ctx: InstructionsContext): ResolvedInstructi
     tocEntries.push({ label: "EVENT CONTEXT", description: "related PR/issue data" });
   tocEntries.push({ label: "SYSTEM", description: "persona, security, tools, workflow rules" });
   if (pctx.learningsFilePath)
-    tocEntries.push({ label: "LEARNINGS", description: "repo-specific knowledge file path" });
+    tocEntries.push({
+      label: "LEARNINGS",
+      description: "repo-specific knowledge file path + heading TOC",
+    });
   tocEntries.push({ label: "RUNTIME", description: "environment metadata" });
 
   const toc = buildToc(tocEntries);
@@ -433,6 +475,7 @@ export function resolveInstructions(ctx: InstructionsContext): ResolvedInstructi
     eventContext,
     system,
     learningsFilePath: pctx.learningsFilePath,
+    learningsHeadings: pctx.learningsHeadings,
     runtime: pctx.runtime,
   });