PR summary as agent-edited tmpfile snapshot (#568)

* PR summary as agent-edited tmpfile snapshot

Replaces the comment-based PR summary path (and the in-progress
update_pr_summary tool from #534) with a snapshot file the agent edits in
place during Review / IncrementalReview / pr-summary Task runs.

The server seeds the tmpfile with the previous snapshot (incremental) or a
stable scaffold (first run), exposes the path via select_mode, and reads it
back at end-of-run to persist to WorkflowRun.summarySnapshot and (when the
prSummaryComment toggle is on) splice into the PR description body.

Why a tmpfile rather than a tool call: incremental snapshot edits are
output-token-cheap when the agent uses native file-editing tools, and
range-diff cleanly across runs because section headings are stable. The
agent never has to regurgitate the full snapshot to update it.

Gating: snapshot generation is opt-in via either prSummaryComment="enabled"
(splice into PR body) or prReReview="enabled" (snapshot feeds future
incremental review runs as context). Users who disable both pay nothing
end-to-end — no seeding, DB write, or body splice.

Behavior changes:
- Drop the Summarize mode and the Summary comment type entirely; the
  rolling summary is no longer a separate run shape.
- pull_request_synchronize with re-review off and summary on still
  dispatches a silent pr-summary Task, but it edits the snapshot file
  instead of posting a fresh comment.
- /api/repo/.../pr/.../summary-comment now returns
  { snapshot: string | null } from the DB instead of fetching a comment via
  GraphQL. URL kept stable so deployed older actions degrade gracefully.
- summaryCommentNodeId is retained on WorkflowRun for legacy data and a
  future backfill of pre-snapshot comment-based summaries.

Supersedes #534. The commit-tool/sub-agent direction in that PR is
abandoned in favor of this file-based shape.

* address review pass #1: synchronize fallback, splice idempotency, docs

* address review pass #2: in-flight skip should not race summary fallback

* address review pass #3: signal-handler flush, doc clarifications

* address review pass #4: in-flight persist promise + bounded body-splice timeout

* address review pass #5: defensive catch on persist worker, doc nit

* add summary-stale post-run gate

When generateSummary is set, we capture the bytes of the seeded snapshot
file and pass them to the agent's post-run loop alongside the file path.
After each agent attempt, the loop diffs the current file against the
seed; if they're byte-identical the agent never touched it, and we nudge
once via a resume turn (similar to the dirty-tree gate, but soft and
fire-once so smaller models that legitimately decide no edit is warranted
don't burn the retry budget).

Mostly defends against forgetful smaller models on the Review path —
their mode prompt asks them to edit the snapshot file, but the
multi-step instruction can fall through when the diff is large.

* trigger: retry vercel preview build

* fix(action): drop unused re-export that pulled node:fs/promises into next bundle

action/internal/index.ts was re-exporting DEFAULT_PR_SUMMARY_INSTRUCTIONS
from action/utils/prSummary.ts, but nothing in the next.js app imports
it. prSummary.ts uses node:fs/promises, and pullfrog/internal is aliased
into the next bundle by next.config.ts, which made turbopack try to
resolve node:fs/promises in client chunks and fail with:

  the chunking context (unknown) does not support external modules
  (request: node:fs/promises)

drop the re-export — selectMode.ts (the only real consumer) already
imports it directly from action/utils/prSummary.ts.

* firewall PR summary snapshot from user instructions; resurrect rich format for Review

The agent-internal snapshot (the markdown file the agent edits in place across
runs) is exclusively durable context for future agent runs — user-supplied
summarization instructions warp it and degrade that context. Drop the
prSummaryCommentInstructions read path end-to-end:

- handleWebhook: stop reading prSummaryCommentInstructions, stop passing
  prSummaryInstructions through dispatch options
- action payload + ToolState + selectMode addendum: drop the instructions
  appendix; the snapshot prompt is fixed, not user-shaped
- TriggersSettings: drop the InstructionsEditor for prSummaryCommentInstructions
- prSummary.ts: reframe DEFAULT_PR_SUMMARY_INSTRUCTIONS as agent-targeted
  (durable context, not human-facing prose)

Prisma columns (prSummaryComment, prSummaryCommentInstructions) and the
matching zod schema entry stay for graceful retreat.

Separately, resurrect PR_SUMMARY_FORMAT (deleted along with the Summarize mode
in the original PR) and wire it into Review mode only. Initial PR reviews now
include a structured summary section in the review body using the rich format
(TL;DR, key changes, ## sections with before/after, file-link trails).
IncrementalReview keeps its existing terser bullet-list shape since re-review
bodies are deltas, not introductions. The user-facing review summary and the
agent-internal snapshot are deliberately separate artifacts with separate
prompts and zero shared content.

* address review comments: prompt self-consistency + stale-doc cleanup

PR 568 self-review (4232488109) flagged a self-contradiction the firewall
commit introduced and three stale doc references that survived.

- action/modes.ts: Review-mode step 2's trivial-PR shortcut said `submit
  "Reviewed — no issues found." per step 5`, but step 5's rewrite removed
  exactly that preamble. Aligned both: trivial PRs and no-actionable-issues
  PRs now produce a body that opens with "No new issues found." followed by
  the PR summary, so the user gets the headline up front and still sees what
  was reviewed.
- docs/pr-reviews.mdx: dropped the "customize the summary style with Summary
  instructions in the console" sentence (the editor was removed in the
  firewall commit). Replaced with a note that the snapshot uses Pullfrog's
  built-in format and is not user-customizable.
- wiki/prompt.md, wiki/modes.md: rewrote the snapshot-prompt entries to
  reflect the firewall — DEFAULT_PR_SUMMARY_INSTRUCTIONS is the entire
  prompt, prSummaryCommentInstructions is no longer wired in.

* drop orphaned prSummaryCommentInstructions column

Prod audit (455 repos): 5 non-null rows on a single account, all containing the
literal placeholder text from the InstructionsEditor we removed in the firewall
commit. No account has an intentional preference set, so silent-ignore (the
keep-for-retreat option) costs us nothing meaningful while leaving an orphan
column in the schema. Drop it.

- prisma/schema.prisma: remove the column
- prisma/migrations/20260506000000_drop_pr_summary_comment_instructions:
  ALTER TABLE ... DROP COLUMN
- utils/schemas/triggers.ts: drop the matching zod entry

* drop body splicing; snapshot is internal-only

User-visible PR summarization continues to ship in Review and IncrementalReview
review bodies (which already render PR_SUMMARY_FORMAT and "Reviewed changes"
respectively). The snapshot tmpfile is now purely durable cross-run agent
context — seed, edit-in-place, save to DB, feed the next run. Massive
simplification: the body splice mechanics, the two-toggle gating matrix, the
summaryHandlingCovered race tracking, and the synchronize summary-only Task
fallback all go away.

Code:
- prSummary.ts: drop splice/strip/marker code (`splicePrSummary`,
  `stripExistingSummaryBlock`, `buildSummaryBlock`, `extractPrSummary`,
  PULLFROG_SUMMARY_START/END). keep scaffold, instructions, seed/read.
- main.ts: rename persistAndPostSummary -> persistSummary; collapse to a
  single DB PATCH. drop pulls.get/pulls.update, drop AbortSignal timeout,
  drop in-flight promise machinery, drop prSummaryToBody plumbing.
- ToolState: add summarySeed (replaces local var in main.ts so persist can
  compare). drop prSummaryToBody and summaryPersistInFlight.
- persistSummary now compares against the seed and skips the DB write
  with a warning when unchanged — saving the seed verbatim is either a
  no-op or persists the placeholder scaffold, neither useful.
- postRun.ts: when summary-stale is the only failing gate and the resume
  turn itself fails, restore the pre-resume successful result and break.
  symmetric with the existing reflection-failure preservation. summary-stale
  can no longer flip a successful run to failed.

Webhook:
- pull_request_opened: generateSummary follows prReReview only (the snapshot
  has no consumer when re-review is off).
- pull_request_synchronize: collapses to "if prReReview enabled, dispatch
  IncrementalReview". the summaryHandlingCovered flag, the same-SHA/in-flight
  coordination it was protecting, and the summary-only Task fallback all
  delete cleanly.

UI / config:
- drop SummarizePRsTrigger (the toggle gated body splice; with that gone
  it has no behavior). drop sidebar entry, console import, Text icon import.
- drop prSummaryComment from triggers zod schema, prisma schema, preview
  settings script.

Migration: squash the two existing migrations into one timestamped
20260507000000_pr_summary_snapshot covering all three column changes
(add summarySnapshot on workflow_runs, drop prSummaryCommentInstructions
and prSummaryComment on repos). repo convention is one migration per PR.

Action: bump 0.0.203 -> 0.0.205 (payload contract changed: prSummaryToBody
removed; main is at 0.0.204).

Out-of-diff cleanup:
- review.ts:190 + review.test.ts:651 — "Reviewed — no issues found." ->
  "No new issues found." to match the canonical body in modes.ts.

Verified: pnpm typecheck clean, pnpm lint clean, postRun + review tests
pass, dev DB reset against production and the squashed migration applied
cleanly (summarySnapshot present, prSummaryComment / prSummaryCommentInstructions
both gone).

* re-orient snapshot toward functional summary; drop prior-review-feedback section

Empirical audit on preview-568 PR #5 showed the snapshot IS load-bearing
for the orchestrator: lens-dispatch prompts on incremental runs carried
forward context from the snapshot's risk register (e.g. "the JSDoc
explicitly scopes to code points — do not flag grapheme-cluster issues"
on the surrogate-pair fix run, "consistency with native padStart" on the
padStart-added run). The orchestrator was reading the snapshot, reasoning
about it, and using it to anti-prime / focus subagents — exactly the
high-leverage path. My earlier "snapshot is write-only" claim was wrong.

The shape, however, was steering it toward review-history-log instead of
functional summary. This commit re-orients:

- prSummary.ts: replace the four-section scaffold (~580 chars of placeholder
  italics under "What this PR does / Key changes / Risk / Reviewed in prior
  runs") with a minimal seed (~150 chars: just a header + a one-line
  comment about what the file is for). different PRs warrant different
  organization; forcing a refactor and a feature into the same template
  is procrustean. minimal seed also makes the unchanged-from-seed gate
  in persistSummary more sensitive.

- selectMode.ts addendum: rewrite around three principles. (1) the snapshot
  is a FUNCTIONAL summary of what the PR does and the risks it carries,
  not a chronological review log — commit history can already be
  reconstructed from list_pull_request_reviews. (2) the orchestrator should
  USE the snapshot during triage and dispatch — concrete example given of
  carrying snapshot context into subagent lens prompts. (3) structure is
  the agent's call; stable headings make snapshots range-diff cleanly when
  they fit, but riff when they don't.

- modes.ts IncrementalReview: drop the "Prior review feedback" checklist
  from the user-facing review body (step 6b gone, step 7 ELSE IFs cleaned
  up). It duplicated content that's already covered by the Reviewed-changes
  bullets and tracked durably in the snapshot for the next agent run; in
  the user-facing body it was noise. step 3 still fetches prior reviews
  but its role is now just filtering aggregation in step 5, not rendering.

- AGENTS.md: codify "no follow-ups" rule. when an issue is identified
  during code review, fix it in this PR — PR scope does not constrain
  quality. follow-up TODOs are forbidden as a substitute for doing the
  work now.

Empirical evidence supporting the re-orientation:

- Run 25568912293 (PR#5 incr1, surrogate-pair fix): orchestrator's
  correctness lens dispatch said "Do NOT flag grapheme-cluster issues
  — the JSDoc scopes to code points." The grapheme-cluster framing was
  not in the diff; it was downstream of the snapshot's prior risk-section
  framing of truncate's contract. Snapshot influencing dispatch.

- Run 25569054779 (PR#5 incr2, padStart added): orchestrator's correctness
  lens dispatch enumerated edge cases including "consistency with native
  String.prototype.padStart contract" and "fill = multi-code-point string
  (e.g. emoji)". Both threads carried over from the snapshot's prior
  truncate code-point-vs-code-unit discussion. Snapshot informing the
  shape of what was looked for.

The cost of maintaining the snapshot (~800 tokens, ~$0.005/run) is
trivially affordable when it materially improves orchestrator triage
on the 1-5 lenses dispatched per review.

utils/instructions.ts

@@ -289,7 +289,7 @@ When embedding images (e.g. uploaded screenshots) in comments or PR bodies, alwa
 
 **\`report_progress\`**: call this exactly once at the end of every run with a brief final summary (1-3 sentences) unless the mode guidance instructs otherwise. Never call it for intermediate status updates (e.g., "Checking for changes...", "Starting review...") — the task list handles live progress automatically. Calling \`report_progress\` replaces the task list with your summary and preserves the current task list in a collapsible section. Keep the summary concise — do not repeat what the task list already shows. Focus on the outcome (what was accomplished, links to artifacts) rather than listing individual steps. If something failed, include the tool's error text even when that makes the summary longer.
 
-Never use \`create_issue_comment\` for task progress — that creates duplicate comments and leaves the progress comment stuck in its initial state. \`create_issue_comment\` is only for standalone comments unrelated to your current task (e.g., Plan comments, PR Summary comments).
+Never use \`create_issue_comment\` for task progress — that creates duplicate comments and leaves the progress comment stuck in its initial state. \`create_issue_comment\` is only for standalone comments unrelated to your current task (e.g., Plan comments).
 
 ### If you get stuck
 

utils/patchWorkflowRunFields.ts

@@ -14,7 +14,8 @@ export type WorkflowRunArtifactPatchKey =
   | "issueNodeId"
   | "reviewNodeId"
   | "planCommentNodeId"
-  | "summaryCommentNodeId";
+  | "summaryCommentNodeId"
+  | "summarySnapshot";
 
 /**
  * Usage fields — aggregated across all agent calls and PATCHed once at
@@ -38,6 +39,7 @@ const STRING_KEYS: WorkflowRunArtifactPatchKey[] = [
   "reviewNodeId",
   "planCommentNodeId",
   "summaryCommentNodeId",
+  "summarySnapshot",
 ];
 
 const NUMBER_KEYS: WorkflowRunUsagePatchKey[] = [

utils/payload.ts

@@ -27,6 +27,7 @@ export const JsonPayload = type({
     id: "string",
     type: "'issue' | 'review'",
   }).or("undefined"),
+  "generateSummary?": "boolean | undefined",
 });
 
 // permission levels that indicate collaborator status (have push access)
@@ -160,6 +161,7 @@ export function resolvePayload(
     timeout: inputs.timeout ?? jsonPayload?.timeout,
     cwd: resolveCwd(inputs.cwd),
     progressComment: jsonPayload?.progressComment,
+    generateSummary: jsonPayload?.generateSummary,
 
     // permissions: inputs > repoSettings > fallbacks
     push: inputs.push ?? repoSettings.push ?? "restricted",

utils/prSummary.ts

@@ -0,0 +1,78 @@
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import { dirname, join } from "node:path";
+
+/**
+ * The PR-level summary snapshot is a markdown file the agent edits in place
+ * during a Review / IncrementalReview run. The server seeds the file with
+ * either the previous run's snapshot (incremental) or a stub scaffold (first
+ * run), lets the agent edit it with its native file-editing tools, then
+ * reads it back at end-of-run and persists it to `WorkflowRun.summarySnapshot`.
+ *
+ * The snapshot is an internal artifact — it is consumed by future agent runs
+ * as durable cross-run context, not surfaced to humans. User-visible summary
+ * content lives in the Review / IncrementalReview review bodies, governed by
+ * `action/modes.ts`.
+ *
+ * Edit-in-place avoids the output-token tax of a tool call that regurgitates
+ * the full snapshot, and gives incremental runs a clean surface that
+ * range-diffs cleanly across runs because the section headings are stable.
+ */
+
+export const SUMMARY_FILE_NAME = "pullfrog-summary.md";
+
+/**
+ * minimal seed for first-run PRs. just a header + a one-line note about
+ * what this file is for. structure is intentionally NOT prescribed —
+ * different PRs warrant different organization, and the agent should pick
+ * a shape that fits this PR. the agent's prompt (see selectMode.ts
+ * `buildSummaryAddendum`) carries the actual instructions for what to
+ * capture and how.
+ *
+ * keeping the seed short also makes the unchanged-from-seed gate more
+ * sensitive — any meaningful edit moves the file off the seed, so
+ * `persistSummary` can reliably skip the DB write when the agent didn't
+ * touch the file.
+ */
+export const SUMMARY_SCAFFOLD = `# PR summary
+
+<!-- durable cross-run context. edit in place; the next agent run reads this
+     before reviewing new commits. structure however serves the PR best. -->
+`;
+
+const MIN_SNAPSHOT_LENGTH = 60;
+/** PG TEXT can hold ~1GB but a sane cap protects the DB / API payloads. */
+const MAX_SNAPSHOT_LENGTH = 32_768;
+
+export function summaryFilePath(tmpdir: string): string {
+  return join(tmpdir, SUMMARY_FILE_NAME);
+}
+
+/** seed the summary file with previous snapshot (incremental) or scaffold (first run). */
+export async function seedSummaryFile(params: {
+  tmpdir: string;
+  previousSnapshot: string | null;
+}): Promise<string> {
+  const path = summaryFilePath(params.tmpdir);
+  await mkdir(dirname(path), { recursive: true });
+  const seed =
+    params.previousSnapshot && params.previousSnapshot.trim().length >= MIN_SNAPSHOT_LENGTH
+      ? params.previousSnapshot
+      : SUMMARY_SCAFFOLD;
+  await writeFile(path, seed, "utf8");
+  return path;
+}
+
+/** read + validate the summary file written by the agent.
+ * returns null when the file is missing or fails sanity checks. */
+export async function readSummaryFile(path: string): Promise<string | null> {
+  let raw: string;
+  try {
+    raw = await readFile(path, "utf8");
+  } catch {
+    return null;
+  }
+  const trimmed = raw.trim();
+  if (trimmed.length < MIN_SNAPSHOT_LENGTH) return null;
+  if (trimmed.length > MAX_SNAPSHOT_LENGTH) return trimmed.slice(0, MAX_SNAPSHOT_LENGTH);
+  return trimmed;
+}