* cancel + restart workflow run when @pullfrog mention is edited
- add `WorkflowRun.triggeringCommentId` (BigInt?, indexed) so the webhook
handler can find the run that was fired by a given comment
- thread `triggeringCommentId` through `reserveRun` / `triggerWorkflow`
- factor `dispatchMentionRun` out of `issue_comment_created` so the same
shape is reused on edit
- replace the `issue_comment_edited` stub: re-evaluates the trigger gate,
cancels prior runs (`octokit.rest.actions.cancelWorkflowRun` + DB
status='cancelled'), then re-dispatches with a `previousRunsNote`
appended to `eventInstructions` so the agent acknowledges the prior
run/PR/artifacts in its summary
- if the edit removes `@pullfrog`, cancel only (no restart)
Co-authored-by: Cursor <cursoragent@cursor.com>
* thread previousRunsNote via dedicated payload field
user prompt has precedence over eventInstructions, so stuffing the
prior-runs note into eventInstructions made it vanish whenever the
trigger comment contained an @pullfrog mention (which is always for the
edit path). pass it as its own payload field and render it alongside the
user's task so the agent actually sees it.
* delete cancelled run's progress comment on edit-restart
so the issue thread doesn't accumulate "This run was cancelled" stubs
on every edit. only deletes for runs we actively cancel; runs that were
already terminal (e.g. completed before the edit) keep their summary
comment in the thread, and `previousRunsNote` links to it so the new
agent can reference prior work.
post-cleanup is race-safe: the action's `validateStuckProgressComment`
swallows the 404 from the deleted comment and exits cleanly, so the
old run's post step cannot clobber the new run's leaping comment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* also cancel + delete progress comment when triggering comment is deleted
mirrors the edit-removes-@pullfrog path: when an @pullfrog comment that
fired a run is hard-deleted, look up any prior runs by triggeringCommentId,
GH-cancel running ones, and delete their leaping progress comments.
skips trigger-gate re-eval (we're tearing down a run, not firing one) and
performs no restart. reuses the existing cancelRunsForTriggeringComment
helper; the returned previousRunsNote is discarded since no dispatch
follows.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: move cancellation before trigger gate in issue_comment_edited
cancelRunsForTriggeringComment now runs before the triggerEnabled check,
so edits that remove @pullfrog still cancel in-flight runs even when the
repo mention trigger is currently disabled (e.g. for non-collaborators).
* anneal: scope cancel updates per-row + simplify edit gate
- replace blanket updateMany on (triggeringCommentId, repoId) with per-row, status-guarded updates so a parallel handler's freshly-reserved run cannot be clobbered into cancelled by a racing edit delivery.
- drop wasMention/isMention early-break in issue_comment_edited; always run cancelRunsForTriggeringComment (DB is the canonical "did this comment ever trigger a run" source). closes the missing-changes.body.from edge and lets us tear down a still-running prior run even if the admin disabled the mention trigger mid-flight.
- buildPreviousRunsNote returns undefined (not "") when no link lines materialize.
- doc cleanups + wiki/modes.md addendum noting issue_comment_edited / _deleted now drive cancel + restart.
Co-authored-by: Cursor <cursoragent@cursor.com>
* address review feedback on cancel/restart semantics
- guard workflow_run.completed update against status='cancelled' so a
successful-but-uncancellable GH Actions job can't resurrect a cancelled
row (and re-bill it) via the completed webhook.
- bucket only status='completed' runs into `preserved` in
cancelRunsForTriggeringComment; cancelled/failed prior runs have stubs
as their progress comment, not summaries worth referencing.
- emit previousRunsNote for the runId-null cancel case so the restarted
agent always knows when it's superseding a prior dispatch.
- drop the agent-forbidden `gh pr list` hint and soften 'was cancelled'
to 'was signalled to cancel' in the note body.
- post a fallback comment when the edit-path dispatch fails (prior run
already torn down and progress comment already deleted).
- symmetrize the delete-handler's pullfrog guard with the edit handler
(key off hook.comment.user, not hook.sender).
- trim misleading comments on the per-row DB update guard.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
The Review and IncrementalReview prompts unconditionally wrapped any
non-critical review body in `> [!IMPORTANT]`, even for trivial nits or
"rough edge" observations. The result is alert fatigue — full-width
colored callouts dominate the page when the actual finding is a single
JSDoc tweak.
Adds an explicit judiciousness preamble to both Review step 5 and
IncrementalReview step 7, and splits the prior single non-critical tier
into two:
- must-address non-critical (`[!IMPORTANT]`) — gated on real
consequences if shipped (incorrect behavior, missing validation,
regressions the author should fix before merge)
- minor suggestions only (no alert) — single-line nits, doc/comment
polish, defer-able observations, "rough edges"
Critical tier wording also tightened to spell out the bar (`bugs,
security, data loss, broken core flows`).
Co-authored-by: Cursor <cursoragent@cursor.com>
* action: dedupe identical reply_to_review_comment calls within a session
PR #610 reproduced a Kimi K2 stutter where the agent's tool_use surface
showed one `pullfrog_reply_to_review_comment` call but GitHub recorded
two byte-identical POSTs 3s apart, leaving a duplicate response on
`action/mcp/review.ts:14`.
Add `duplicateReplyDecision` (mirrors `duplicateReviewDecision`) and
track per-session replies on `ToolState.reviewReplies`, keyed by
parent `comment_id` + `bodyWithFooter`. Identical re-emissions short
circuit with `{ skipped: true, reason }` instead of POSTing again.
Body-keyed (not just id-keyed) so legitimate follow-up replies with
different content still go through.
Tighten `AddressReviews` step 5 to say *exactly once per comment* and
note that the runtime dedupes identical bodies, so the agent has both
prompt-level guidance and a server-side guarantee.
Co-authored-by: Cursor <cursoragent@cursor.com>
* address review: drop stale file ref in dedupe comment; soften tool description
* remove comment.test.ts
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* claude: surface structured error from is_error result events instead of dumping NDJSON
Co-authored-by: Cursor <cursoragent@cursor.com>
* claude: tighten error-surface fixes (anneal round 1)
Co-authored-by: Cursor <cursoragent@cursor.com>
* claude: remove tests per request
* claude: gate is_error short-circuit on subtype=success, restore error_* branches
* claude: preserve fallback token table for error_* subtypes
the `lastResultError === null` guard was too broad — `error_max_turns` /
`error_during_execution` / `error_*` subtypes set `lastResultError` from
`event.errors[]` and represent runs that genuinely consumed tokens, so
suppressing the fallback table silently dropped billing visibility for
those cases. gate on a dedicated `syntheticStopFailure` flag that's set
only for the `subtype: "success"` + `is_error: true` case where
`accumulatedTokens` is stale.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
`mcp/checkout.test.ts` and `mcp/reviewComments.test.ts` previously hit
live GitHub on every run via `acquireNewToken()`, requiring `GH_TOKEN`
or `GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` in the env. that made them:
- cred-gated — the action runtime filters `_KEY$` / `_TOKEN$` from
subprocess env, so the husky pre-push hook (which runs
`pnpm -r test`) blocked Pullfrog agents from pushing branches. issues
#562, #563, #564, #566 all hit this exact blocker and never got their
fixes pushed.
- non-deterministic and slow (network round-trips for a snapshot test).
both tests are really snapshot tests of pure formatters
(`formatFilesWithLineNumbers`, plus `parseFilePatches` /
`buildThreadBlocks` / `formatReviewThreads` for review data). the live
fetches were just an inefficient way to obtain fixtures.
changes:
1. extract a pure `formatReviewData({ review, threads, prFiles, ... })`
from `getReviewData` in `mcp/reviewComments.ts`. `getReviewData`
becomes thin orchestration: fetch + call formatter. preserves the
"skip listFiles when no threads" perf optimization.
2. add `action/mcp/__fixtures__/` with checked-in JSON captures for the
three fixture test cases (pullfrog/test-repo#1 listFiles,
pullfrog/scratch#49 review 3485940013, pullfrog/scratch#64 review
3531000326). ~14KB total. fixtures store only the fields the
formatter reads — volatile fields (sha, blob_url, etc.) are dropped.
3. rewrite both test files to load the fixtures and call the pure
formatters directly. snapshot keys updated; snapshot content
unchanged (verified by running existing snapshots against the
refactored tests).
4. add `action/scripts/refresh-test-fixtures.ts` to re-fetch the
fixtures from live GitHub on demand:
`node action/scripts/refresh-test-fixtures.ts` (with creds in
`.env` or env). re-run when the GitHub API response shape changes
and review the snapshot diff.
trade-off: a silent change to GitHub's `pulls.listFiles` /
`pulls.getReview` / GraphQL `reviewThreads` response shape would no
longer break this test on every push. that tradeoff is worth it: shape
drift on those endpoints is rare (years between changes), and a
dedicated cron that runs the refresh script and opens a PR on diff is
a far better signal than a flaky cred-gated pre-push hook.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(push_branch): retry transient push errors and surface full stderr/stdout
issue #571 motivated three small improvements to `mcp__pullfrog__push_branch`:
1. classify push errors into `concurrent-push` / `transient` / `unknown`.
- `concurrent-push` extends the existing `fetch first` / `non-fast-forward`
matcher to also catch the server-side `cannot lock ref` form (the case
#571 reports). all three route to the same fetch + integrate + retry
recovery message; copy now mentions concurrent push as a likely cause.
- `transient` covers RPC failed, early EOF, connection reset, dns flake,
HTTP 5xx, HTTP/2 stream not closed, and unexpected sideband disconnect.
these are retried in-tool with 2s + 5s backoff before surfacing the
error. push is idempotent so verbatim retry is safe.
- `unknown` (auth/permission/protected-branch/4xx) is rethrown unchanged —
retrying these wastes time and noise.
2. surface stdout alongside stderr in `$git` failure messages and include the
exit code. previously only `stderr.trim()` was forwarded, which could be
empty in rare HTTPS failure modes (the agent on issue #571's run saw a
one-line `failed to push some refs` and had nothing to diagnose with).
3. unit tests for the classifier covering all three branches plus the
concurrent-push-wins-over-transient ordering.
does not introduce auto fetch+rebase+retry inside the tool — that path is
blocked under shell=disabled, can leave the working tree mid-conflict, and
would create unwanted merge commits. the recovery message keeps the agent
in the loop.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(push_branch): retry 429, jitter backoff, downgrade retry log to info
- treat HTTP 429 (rate-limit / abuse detection) as transient — GitHub
occasionally surfaces it on git push, where it is retry-safe unlike
401/403/404
- add ±25% jitter to backoff so concurrent agents hit by the same
upstream blip don't retry in lockstep
- log retries with log.info instead of log.warning to match retry.ts
convention; a successful retry shouldn't leave a yellow GHA annotation
behind in the job summary
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* cherry-pick updated /anneal command from billing branch + add as Claude Code slash command
mirrors origin/billing:.cursor/commands/anneal.md (commit 4f389a8f) into
both .cursor/commands/ and .claude/commands/ so the parallel-lens annealing
prompt is available in both editors. content is identical between the two
files.
* anneal: drop REVIEW.md pointer, surface-agnostic dispatch wording, fix modes.ts self-review contradictions
Anneal pass over the /anneal slash command and the Build-mode self-review step:
- Drop REVIEW.md references in both anneal.md copies. The file does not
exist on the Claude Code surface (only .cursor/commands/), and its
contents (correctness/security/impact framing) directly contradict the
prescribed single-lens, no-pre-shaping discipline.
- Replace "Task tool calls" with surface-agnostic "parallel subagent
calls" so the meta-prompt does not couple to either CLI's tool naming.
- Hedge the "verify via web search" instruction to acknowledge subagents
may not have web search available.
- modes.ts: drop "and the changed files" — the same step's don't-list
forbids handing subagents a curated reading list (in-file contradiction).
- modes.ts: restore the "skim only, don't pre-review" warning that the
long-form treats as load-bearing.
- modes.ts: drop "NO MCP tools" — overbroad; the actual safety property
is captured by "no writes, no shell commands, no side effects".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal: two-round self-anneal of /anneal + modes.ts self-review
Expand the multi-lens parallel-review protocol with fixes surfaced by
running /anneal on this branch twice. Material additions:
/anneal canonical (.claude/commands/anneal.md + .cursor mirror):
- promote orientation-vs-defect-hunting distinction to a load-bearing
framing in the opening paragraphs
- add an empty-target early exit ("nothing to anneal" stop) at §1
- spell out the read-only constraint with the no-op-if-reverted test,
and forbid recursive subagent dispatch (incl. agentic MCP tools)
- add cleanup-and-debt sub-categories (env vars, feature flags, dangling
symbols), supply-chain, test-integrity lenses to the catalog
- §1 lens-count rule: explicit trivial/typical/high-risk tiers; "treat
as typical" tiebreaker for the unsure case
- §2 example uses bare `git diff <primary-branch>` to capture
uncommitted edits (three-dot syntax is committed-only)
- §5 targeted-follow-up cross-references the fresh-eyes carve-out in
Delegation discipline
- final-message format spells out coverage shape, findings-table
shape, dry-run fix-plan branch, and plan/doc summary branch
- stopping criteria distinguish "trivial" from "small / low-risk"
action/modes.ts Build mode step 4 (self-review one-pass anneal):
- empty-diff early exit; "step 4 mandatory whenever there is a diff"
resolves the prior contradiction with the always-runs assertion
- lens count by risk (2-3 typical / 4 high-risk single-round-cap /
exactly 1 trivial) with separate Tiebreaker
- expand swap-in lens menu (research-validated assumptions, security,
user-journey, ops, integration, test integrity, supply chain,
performance, holistic) so the catalog is a starting menu, not a
closed set
- rename `cleanup & scope` to `diff hygiene` to avoid colliding with
the canonical's broader `cleanup & debt`
- delegation discipline bulletized (don't lens-review yourself,
don't summarize, don't curate, don't pre-shape, don't mention other
lenses); independence rationale stated inline
- explicit research-discipline reminder for any lens that touches
external contracts (web search, quote URLs)
- comment block enumerates deliberate omissions vs the canonical
(dry-run, severity categorization, read-only shell) and the
deliberate scope decision (sibling diff-producing modes stay solo)
action/modes.ts Review + IncrementalReview subagent-dispatch wording:
- propagate the no-recursive-dispatch rule (was missing)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* add set_plan/get_plan + restructure Review/IncrementalReview as parallel-subagent orchestrators
Build mode's self-review and Review/IncrementalReview now follow the multi-lens
parallel-subagent fan-out pattern from the canonical /anneal protocol. New
set_plan/get_plan MCP tools (orchestrator-only) persist the implementation plan
in tool state so the self-review's plan-adherence lens can verify the diff
against the original intent rather than reconstructing it post-hoc.
Subagent "read-only / no further dispatch" is currently enforced via prompt
prose only — neither claude-code's --disallowedTools nor opencode's per-agent
tools allowlist is configured to scope subagent MCP access. Documented as a
deferred ~30-50 LOC follow-up in the modes.ts header comment.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* revert Review/IncrementalReview mode prompts to main; keep Build self-review changes
E2e testing on this branch only exercised the trivial-1-lens path for Review (preview
repo had only docs PRs). Multi-lens Review fan-out was never directly validated against
a real code PR. Splitting the Review/IncrementalReview restructure to its own branch
(review-mode-orchestrator, draft PR #555) pending focused validation.
Keep on this branch:
- set_plan/get_plan MCP tools
- Build mode multi-lens self-review (Test 3 directly validated 2-subagent parallel
fan-out on a 2-file diff)
- /anneal command updates (.claude/ and .cursor/ mirrors)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* require plan parameter when selecting Build mode
Adds an arktype .narrow on SelectModeParams that rejects select_mode({mode:"Build"})
unless a non-empty 'plan' string is also provided. When valid, the plan is stored
into ctx.toolState.plan at mode-selection time, so step 4's plan-adherence lens
always has a comparison target.
This closes the e2e finding that agents never reached for set_plan on their own
(5 of 6 runs in production). Build mode prompt updated to reflect that plan is
already populated at mode selection; set_plan remains as the mid-task replan
tool. Other modes are unaffected.
Validation surfaces the error to the agent with a descriptive message including
the path ('plan') and recovery instructions, so a failing call is recoverable
on the next turn rather than a hard fail.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* move Build-mode plan-required check from arktype .narrow to execute()
arktype .narrow predicates aren't JSON-Schema serializable — FastMCP's
toJsonSchema() emitted a {code: "predicate", predicate: Function} object
instead of a serialized schema. Effect: agents couldn't see select_mode
in their tool list (verified by 5 consecutive runs across two models
silently bypassing select_mode entirely after the prior commit).
Fix: keep the param schema clean (.narrow removed) and check
selectedMode.name === "Build" && !params.plan in the execute() body,
returning a structured error response. The agent now sees select_mode
normally, gets a clear actionable error if it forgets the plan, and can
recover on the next turn by retrying with the plan included.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* flip lens architecture: Build = single fresh-eyes subagent, Review/IncrementalReview = multi-lens
Build mode self-review previously fanned out 1-4 lenses on the agent's own diff. The
bias-mitigation argument for fan-out is weaker for self-review than for reviewing
someone else's PR — the orchestrator just wrote the code, so what matters is one
fresh-eyes subagent that doesn't share the implementation context, not breadth across
parallel angles. Build now dispatches exactly one subagent that gets the original
user request and the diff and evaluates whether the diff fulfills the request.
Review and IncrementalReview now use the multi-lens orchestrator pattern (triage →
parallel read-only fan-out → aggregate → draft comments → submit). For someone else's
PR, parallel lenses (correctness, security, research-validated, user-journey, etc.)
provide breadth that a single subagent can't carry coherently. Was previously parked
on the review-mode-orchestrator branch (PR #555).
Removes set_plan/get_plan MCP tools, ToolState.plan field, and the plan parameter on
select_mode. Validated end-to-end that those didn't cause agents to actually use plan
tracking (5 of 6 e2e runs skipped them); the original user request from the prompt
body is the source of truth and the orchestrator already has it.
Drops timeout test plan-param workaround that was added for the prior validation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* split Review/IncrementalReview multi-lens back out to review-mode-orchestrator branch
The multi-lens orchestrator restructure for Review/IncrementalReview was bundled
into this branch in commit e964ae0c, but it hasn't been validated against a
real code-heavy PR (the e2e exercised it only on docs PRs). Splitting it back
out keeps this branch focused on the validated half — Build → single fresh-eyes
subagent — and lets the Review changes ship in a focused PR (#555 reopened).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal: fix Build prompt contract bugs found by 3-lens review
Major fixes:
- checkout_pr returns the field as `base`, not `baseRef` (per checkout.ts:611-616).
The prompt was telling agents to read `result.baseRef` which would be undefined.
- The base-ref fallback "after fetching" is unreachable via the `git` MCP tool
(it blocks `fetch` per AUTH_REQUIRED_REDIRECT). Now names `git_fetch` explicitly.
- Boundary-tag wrapping for the user request had no escape rule for input that
contains the literal close marker, and no fallback for an empty request. Both
are now documented with a nonce-suffix mitigation.
- PR reference updated #555 → #557 (the active PR for the multi-lens
review-mode-orchestrator branch; #555 was closed after the rebase).
Minor fixes:
- Retry predicate tightened: "errors out (tool error) or returns an empty body",
not "returns nothing usable" (which is unfalsifiable and lets an orchestrator
declare any output not-usable to skip review).
- Subagent read-only constraints rephrased as prescriptive ("MUST NOT call")
rather than descriptive ("you have only"), since on inheriting runtimes the
subagent does in fact have access to write tools and the constraint is
prompt-only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 2: tighten Build prompt edge cases (workflow_dispatch, base-ref, footer-strip, skip marker)
Cross-lens findings from holistic + user-journey + research-validated lenses:
- workflow_dispatch + empty diff: report_progress silently no-ops when there's
no parent issue/PR. Now also call set_output with a "no-op" summary so the
user gets surfacable feedback.
- base-ref resolution: clarified `base` from checkout_pr is a bare ref name,
added explicit `git remote show origin` path for repos whose primary is not
`main` (master, trunk, etc.).
- bare `git diff` description: tightened from "shows working tree" to
"shows unstaged working-tree changes" — bare diff misses staged changes too,
not just committed ones.
- prompt-body stripping: explicitly call out the leading `> ` blockquote
prefix (added by the *YOUR TASK* section formatting) and the entire Pullfrog
footer block, not just one example link.
- boundary-tag nonce: always-on now, not conditional on detecting a close
marker. Cost is one random short string; failure mode (prompt injection if
input contains literal close marker) is silent.
- subagent-skip marker: structured `Self-review: SKIPPED (subagent error: ...)`
on its own commit-message line, so the gap is greppable.
Header comment also documents:
- AddressReviews/Fix/Task asymmetry (deliberately deferred)
- Subagent-runtime-fence deferred fix must explicitly deny Skill / agentic
MCP tools, not just destructive tools (claude-code blocks recursive Task
spawn but not alternative dispatch paths).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 3: targeted re-review of round-2 changes catches real regressions
Round 2's "fixes" introduced two real bugs that round 3's targeted correctness
re-review caught:
CRITICAL (fixed): tier-3 base-ref resolution used `git remote show origin`,
which requires network auth — the MCP `git` tool runs commands through plain
spawn() without auth, so this hangs on private repos. Replaced with
`git symbolic-ref refs/remotes/origin/HEAD` (local symref, no network),
which actions/checkout populates.
MAJOR (fixed): the eventInstructions fallback was incoherent — the agent has
no separately-addressable eventInstructions field; whatever it received in
*YOUR TASK* is its only input. Removed the misleading reference.
MAJOR (fixed): per-line `> ` strip was ambiguous, could destructively flatten
user-pasted markdown blockquotes. Now: "strip exactly one leading `> ` per line".
MAJOR (fixed): tier-1 base-ref preferred bare `<base>` over `origin/<base>`,
which fails on the rare alreadyOnBranch path in checkout_pr where the local
ref isn't re-created. Now prefers `origin/<base>` (always populated post-fetch).
MINOR (fixed): footer-strip anchor was `<sup>`/`<picture>`, both of which
appear in legitimate user content (footnotes, etc.). Switched to the
PULLFROG_DIVIDER sentinel which is purpose-built for this.
MAJOR (acknowledged, partial fix): 4-hex nonce is theatrical security; bumped
to 8 hex and explicitly noted it's a typo-guard, not a security boundary,
and that the structural fix (separate task() argument) is the real solution.
REJECTED (verified false positive): subagent claimed `set_output` is not
registered for workflow_dispatch. Verified at action/utils/payload.ts:118 —
workflow_dispatch from `gh workflow run` resolves to trigger:"unknown",
which IS standalone, which IS registered with set_output. E2e logs from
prior tests confirm agents successfully call pullfrog_set_output on
workflow_dispatch runs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 4: drop broken symbolic-ref tier, simplify base-ref resolution
Round 3's tier-2 (`git symbolic-ref refs/remotes/origin/HEAD`) is
empirically broken: actions/checkout doesn't populate origin/HEAD on
shallow clones (fetch-depth: 1, used by pullfrog.yml), and Git 2.50+
no longer auto-sets it on full clones either (actions/checkout#2219).
New scheme: PR context uses checkout_pr's `base`. Non-PR context tries
origin/main first; if that fails, list remote branches with
`git branch -r` and pick the obvious default (master/trunk/etc.).
Drops the symbolic-ref path entirely (broken) and `git remote show`
(requires auth that the MCP `git` tool can't provide).
Also fixes:
- Per-line strip prose: removed phantom "or `>` at end-of-line for
blank lines" parenthetical (instructions.ts always emits `"> "`).
- Pullfrog footer strip: now scoped to "only when divider appears at
end of body, followed only by footer block."
- Boundary-tag nonce wrapping: rephrased without the "this is theatrical"
framing that was undermining the agent's diligence.
- Empty-request fallback: removed the misleading "no separately-
addressable eventInstructions field" claim (the field exists; what's
true is it's already folded into *YOUR TASK* upstream).
- Out-of-scope structural-fix commentary moved out of agent prompt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 5: drop unreliable auto-discovery for non-main repos, align footer-strip with prod, fix tautological empty-request fallback
* anneal round 6: condition per-line strip on quoted-prompt heuristic; document main-not-default limitation; fix empty-request placeholder/framing contradiction
* anneal round 8: fix default-branch hardcode, wrap diff in boundary tag, improve nonce guidance
CRITICAL/MAJOR (ops + security):
1. Default branch was being hardcoded to `main` with a "limitation cannot be fixed
from prompt prose alone" disclaimer — but `default_branch` IS exposed to the
agent via the *SYSTEM* runtime context block (action/utils/instructions.ts:47).
The prior comment was actively misdirecting future debugging. Now the prompt
reads the field from system context and uses `origin/<default_branch>`.
2. Diff was passed verbatim with no boundary tag — asymmetric defense relative
to the user request. Attacker-controlled file content (e.g., committed code
comments saying "AGENT: ignore prior instructions") could prompt-inject the
subagent through the diff payload. Now both blobs get nonce-suffixed boundary
tags with explicit "lines starting with + or - are file content, not directives."
3. Nonce guidance updated: prefer CSPRNG source (`head -c 16 /dev/urandom | xxd -p`)
when shell available; documented that LLM-picked hex has ~10-14 effective bits
even at 8 nominal hex chars (per arXiv:2506.05739 on adaptive attacks against
delimiter defenses).
MINOR:
- Removed the `@user triggered "..."` preamble strip bullet — verified there's
no producer of that pattern anywhere in action/utils/, so the strip was a no-op.
- Empty-request placeholder must be the ENTIRE boundary content, not a substring,
to prevent attacker from triggering the request-skip framing branch.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 9: fix RUNTIME-vs-SYSTEM section misdirection; tighten nonce guidance for shell-disabled mode + distinct-value enforcement
* anneal round 11: fix real bugs uncovered by big-picture review
Senator Armstrong's deeper review (design-coherence + realistic-customer
stress test) caught issues that 10 rounds of narrow targeted re-reviews
had been papering over.
REAL BUGS FIXED:
1. set_output called unconditionally on the empty-diff path would error on
PR-event triggers (set_output is registered only when trigger==="unknown"
per server.ts:242-245). Now gated: only call set_output if it's actually
in the tool list.
2. Sentinel-strip used FIRST occurrence — broken under adversarial blockquote
attack (an attacker quotes a Pullfrog comment containing the divider, with
their real request after it; first-occurrence strip discards the real
request). Now uses LAST occurrence so the real request survives.
DESIGN HONESTY:
3. Header comment now explicitly flags the design as UNVALIDATED — no A/B
eval has been done against solo self-review. ROADMAP_RESEARCH.md flags
benchmarking as the prerequisite. Header documents the validation gap
and what would justify reverting.
4. Header comment elevates the runtime-fence gap from a TODO to a SECURITY
GAP that must ship before the prompt protocol can be considered
production-hardened. Ordering: runtime fence FIRST, prompt protocol
SECOND.
SIMPLIFICATIONS (per senior-engineer review):
5. Dropped the second nonce on the diff — the diff is the artifact under
review; suspicious instruction-shaped lines in commits are exactly what
the subagent should flag, not something to fence off.
6. Dropped CSPRNG-vs-LLM-fallback branching prose — just "16+ hex chars,
use /dev/urandom if shell available, otherwise pick."
7. Dropped the regenerate-if-collide rule (vanishingly unlikely with 16
hex chars, costs tokens to enforce).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* anneal round 12: revert round-11 regressions (sentinel-strip, set_output gate, diff nonce)
Round 12's sharper review caught three regressions round 11 introduced:
1. Sentinel-strip last-occurrence was strictly worse than first-occurrence
for the common "user references a prior Pullfrog comment" case. The
adversarial-quote scenario it was defending against is contrived (an
attacker can put hostile payload anywhere; strip discipline doesn't
change attack surface). Reverted to first-occurrence to align with
canonical stripExistingFooter() and avoid silently swallowing user
reference context.
2. set_output "gate" via "if it's in your tool list" relied on tool
introspection that LLMs cannot reliably perform. Replaced with: just
call report_progress; document the workflow_dispatch limitation as
acceptable (job log is feedback-of-last-resort) rather than asking the
agent to conditional-call a tool that may not exist.
3. Diff was de-nonced in round 11 on the assumption runtime fence ships
first, but until that runtime fence lands the plain label is forgeable
(committed file content can include "--- END DIFF ---" + injection).
Restored nonce wrapping. The cost is one extra hex string; the benefit
is real until runtime fence ships.
Also added explicit caveat on the self-attested skip marker: the proper
fix is MCP-layer dispatch-counting, not commit-message annotation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* ruthless cut: revert Build self-review elaboration to compact form
main already had subagent dispatch (4 compact lines). This branch added 70+ lines
of elaboration — header warnings, base-ref dance, footer-strip rules, nonce-
suffixed boundary tags, retry-once skip markers, delegation-discipline list — all
predicated on a runtime fence that doesn't exist and validation that never ran.
Senior-engineer review (round 11) explicitly recommended cutting; ROADMAP_RESEARCH
flags A/B benchmarking as the prerequisite for this design.
Net change vs main now matches what the user actually asked for:
- drop the optional plan step (and its "follow the plan" / Notes references)
- subagent receives the original user request alongside the diff, evaluated
against base ref, with explicit no-further-dispatch constraint
Everything else reverts to main's prose. ~10 lines net change instead of 70+.
* anneal round 13: tighten self-review prompt inputs to runtime-resolvable values
Two underspecified inputs flagged by parallel holistic + mechanics review:
1. "the original user request" is empty for non-@pullfrog-tagged auto-triggers
(sync, check_suite, opened, etc.); only YOUR TASK is reliably present in
the assembled prompt across all event types. Replace.
2. "base ref (PR base or repo default branch)" requires the agent to resolve
and fetch the default branch on non-PR runs (origin/<default> typically
not fetched). Drop the elaboration — bare git diff captures all changes
at step-3 time since step 2 doesn't commit. Aligns with 3ed2c55a's
ruthless-cut philosophy: less elaboration, not more.
Verified in round 14: YOUR TASK is the literal section header in
instructions.ts (buildTaskSection); bare git diff scope is correct.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* restore plan step to Build mode prompt
The plan step was removed alongside the MCP-contract plan-required work,
but the user only wanted it gone from the MCP contract, not from the
prompt itself. Restores step 1 (plan), the "follow the plan" build
sub-bullet, the trailing Notes section, and renumbers learningsStep
back to 6.
Made-with: Cursor
* add pullfrog-reviewer named subagent; standardize review fence to non-mutative+non-recursive
Defines a constrained `pullfrog-reviewer` named subagent for the Build
mode self-review and /anneal lens dispatch, with a single source of
truth in action/agents/reviewer.ts (allowed tools, denied mutating MCP
tools, system prompt).
Enforcement:
- opencode: real fence via agent.pullfrog-reviewer block in
buildSecurityConfig — denies edit/bash/task and globs each mutating
pullfrog_* MCP tool to false.
- claude-code: forward-looking only. Per-agent disallowedTools is
upstream-broken (anthropics/claude-agent-sdk-typescript#172, open as
of latest update Mar 2026 — subagent child processes still see and
can call disallowed tools, including Task). The --agents JSON is
defined anyway so the fence becomes real when upstream fixes#172;
until then the prompt prose constraint is the actual fence. The
PreToolUse hook workaround that does enforce is out of scope.
Read-only MCP tools (get_*, list_*) intentionally remain enabled so
the reviewer can pull PR/issue/check context without dispatching
state changes.
Both modes.ts Build self-review and the two anneal.md files now share
the same "non-mutative + non-recursive" framing — file reads, grep,
search, web search/fetch, read-only shell, and read-only MCP queries
allowed; writes, state-changing MCP, and nested subagent dispatch
denied. Resolves the previous inconsistency where /anneal allowed
read-only shell and Build self-review banned all shell.
Made-with: Cursor
* Build self-review: pass build-phase failure summary to reviewer subagent
Adds an instruction in step 4's dispatch: along with YOUR TASK and
git diff, pass a tight plain-text summary of any lint/typecheck/test
failures fixed during build (what broke, root cause, the fix) — or
"no build-phase failures" if clean. Goal: let the reviewer check
that fixes addressed root causes rather than suppressed symptoms
(e.g., editing a test to make it pass instead of fixing the bug).
Implemented as agent self-summarization rather than piping raw build
output to avoid context flooding — typecheck/test output can be
hundreds to thousands of lines per failure. The agent has the
failure trail in its own conversation history and summarizes from
memory; the reviewer sees a few lines per failure, not raw stderr.
Caveat: this is a plausible-but-unvalidated quality improvement.
The mechanical justification (signal already produced, currently
not passed on) is real; "this catches more bugs" is a hypothesis
that will need actual run data to confirm. Downside is bounded
(reviewer gets slightly more context, no behavior change if the
summary is empty or ignored).
Made-with: Cursor
* Build self-review: distill /anneal delegation + research discipline into dispatch instructions
Lifts the codified learnings from /anneal's "Delegation discipline" and
"Research discipline" sections into Build mode step 4. These rules are
about how-to-prompt the reviewer (not about parallelism), so they
transfer losslessly to single-agent dispatch and address bias modes the
prior prompt was silent on:
- Don't summarize what you implemented (biases toward shape-validation)
- Don't curate a reading list (your curation is itself a lens)
- Don't pre-shape output with severity/category (leaks hypotheses)
- Don't defect-hunt in parallel (reintroduces the implementation bias
the subagent is meant to mitigate)
- For diffs touching third-party API contracts / SDK semantics /
framework directives / DB engine specifics, instruct the reviewer to
verify load-bearing claims via web search and quote URLs rather than
trust training data
Restructures step 4 from one paragraph into three (constraints, inputs,
discipline) plus a final review-and-commit paragraph for readability.
These are validated learnings from many anneal rounds, not theoretical
best practices — they're the single substantive piece this branch was
missing.
Made-with: Cursor
* pullfrog-reviewer: drop MCP deny-list, rely on prose constraint
Per-PR-review feedback: hand-maintaining MUTATING_MCP_TOOLS against
action/mcp/server.ts was fragile — a future mutating tool added to the
MCP server without updating this list would silently grant write access
to the reviewer. Inverting to an allowlist or adding a structural test
both keep the drift problem.
Drop the list and all per-agent runtime denies (claude disallowedTools,
opencode tools/permission map). Strengthen REVIEWER_SYSTEM_PROMPT to
spell out the categories of state-changing MCP tools by example and
explicitly tell the model to apply the no-op-if-reverted invariant to
tools added after the prompt was written — the rule is the invariant,
not the enumeration. Keep the named subagent so the prompt is reliably
injected. Update modes.ts and both anneal.md copies to drop the
runtime-enforces-where-supported claim.
Co-authored-by: Cursor <cursoragent@cursor.com>
* pullfrog-reviewer: fix description to allow read-only shell
The description field was overstating the constraint as 'must not shell',
but the system prompt explicitly allows read-only commands like git diff,
git log, cat, ls. Align description with the actual contract.
Co-authored-by: Cursor <cursoragent@cursor.com>
* restructure Review/IncrementalReview as multi-lens parallel-subagent orchestrators
For someone else's PR, parallel lenses (correctness, security, research-validated
claims, user-journey, etc.) provide breadth across angles that a single subagent
can't carry coherently. The orchestrator does triage → parallel read-only subagent
fan-out → aggregate → draft comments → submit. Lens count by risk: 1 lens for
trivial PRs, 2-3 for typical, 4 for high-risk surfaces (billing, auth, migrations).
This branch contains ONLY the Review/IncrementalReview multi-lens prompts.
Build mode keeps its single-fresh-eyes-subagent shape (different problem —
orchestrator just wrote the code; bias-mitigation comes from one subagent that
doesn't share the implementation context). The Build changes ship in a separate
PR (self-review-subagents → main).
Pending validation against a real code-heavy PR before merge — e2e on a docs-only
preview repo only exercised the trivial-1-lens path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Review/IncrementalReview: dispatch fan-out via reviewfrog named subagent
The fan-out steps previously said "launch one read-only subagent per lens" without naming the
subagent. That bypassed the only enforcement layer the named subagent provides: a baked-in
system prompt that restates the non-mutative + non-recursive contract regardless of what the
orchestrator sends. Both modes now dispatch via REVIEWER_AGENT_NAME (matching Build mode's
self-review wiring) and restate the constraint inline so the rule is present twice.
* rename pullfrog-reviewer → reviewfrog
Mechanical rename of the named subagent. Constant names (REVIEWER_AGENT_NAME, REVIEWER_SYSTEM_PROMPT)
and file paths (action/agents/reviewer.ts) stay as-is — only the agent identifier string and prose
references in anneal.md and code comments change.
* modes/anneal: trivial PRs skip review entirely; lens count is judgment, not table; allow subsystem lenses
Three coupled changes to Review/IncrementalReview/Build self-review and the canonical /anneal
command:
1. Trivial-skip: trivial diffs (single-line, formatting/comment-only, doc typo, low-risk dep
bump, no behavior change) skip the fan-out / self-review entirely. Build mode skips its
self-review subagent; Review submits a bare "Reviewed — no issues found." without
dispatching lenses; IncrementalReview takes the existing non-substantive submit path.
Tiebreaker on uncertainty: treat as non-trivial.
2. Drop prescriptive lens counts. Replaces "2-3 typical / 4 high-risk cap / 1 trivial" with
judgment-based guidance: pick as many lenses as the target has distinct surfaces of risk
worth investigating independently; one is sometimes enough; bias toward more (and toward
follow-up rounds in /anneal) for high-stakes subsystems; 5+ is a smell that lenses are
overlapping rather than covering distinct ground.
3. Subsystem lenses. Adds an explicit second flavor of lens — domain-scoped frames like
"the auth lens", "the billing lens", "the schema-migration lens" — alongside the existing
themed lenses (correctness, security, user-journey, etc.). Stack themed + subsystem freely.
modes.ts and anneal.md (.cursor/ + .claude/, kept byte-identical) move together so the
canonical pattern doc and the orchestrator prompt agree on the protocol.
* add SessionLabeler so parallel subagent log lines are differentiable
When the orchestrator dispatches multiple `reviewfrog` subagents in a single
assistant turn (the parallel fan-out the multi-lens prompt now requires),
their tool_use / tool_result / text events arrive on opencode's NDJSON
stream tagged with distinct `sessionID`s but go through a single
`[Pullfrog]` log prefix. Result: log readers can't attribute which lens
issued which tool call, making CI logs unreadable for any review with 2+
lenses.
SessionLabeler:
- Binds the first-seen sessionID to "orchestrator" and subsequent new
sessionIDs to FIFO-popped lens labels seeded from task tool_use inputs.
- Derives labels from `lens: <name>` markers in the dispatch prompt, the
Task `description` field, the `subagent_type`, or `subagent#N` fallback.
- Keeps state local to a single runOpenCode invocation.
Wiring:
- opencode.ts: every event handler (init, message, text, tool_use,
tool_result) now looks up the per-event label and prefixes log output
via formatWithLabel(). Subagent finalOutput/token-reset paths gated on
ORCHESTRATOR_LABEL so child sessions can't clobber parent state.
- claude.ts: claude rolls subagent activity into a single tool_result
block (no per-event session_id), so it gets a minimal "» dispatching
subagent: <label>" log line on Task tool_use as the only attribution.
- modes.ts (Review + IncrementalReview): orchestrator instructed to set
the Task `description` to the lens name, since that's what the labeler
reads when no explicit `lens:` marker is in the prompt.
Tests: 18 unit tests covering label derivation, FIFO binding, interleaved
sessions, fallback paths, and a realistic four-lens parallel fan-out
simulation. Full action test suite stays green (400 passing).
This is the pre-flight instrumentation that the multi-lens validation
runs depend on — without it, post-hoc log analysis can't tell two
subagents apart.
* log subagent dispatch + finish at info level for per-lens visibility
OpenCode's runtime currently encapsulates subagent execution inside the
`task` tool — subagent-internal tool_use/tool_result events do not surface
on the parent's NDJSON stream. The SessionLabeler I added in 0c4647f4
therefore can't actually differentiate concurrent subagent log lines
(there are no concurrent log lines on the parent stream to differentiate).
What CAN be observed on the parent stream is the dispatch and the result
of each `task` tool call. This patch surfaces both at info level:
» dispatching subagent: lens:security (subagent_type=reviewfrog)
...
» subagent finished: lens:security (15.3s, status=completed) — ...
Without this, a 4-lens parallel fan-out looks like 4 dispatches in close
succession followed by a long quiet gap and then an aggregation turn —
you can't see when each lens finished or how the durations overlapped.
With it, parallel execution is visible from the timestamps on the
"finished" lines.
The dispatched label comes from SessionLabeler.recordTaskDispatch (so
both lines share the same lens identity). taskDispatchInfo maps callID to
{label, startedAt} so the matching tool_result can compute duration and
emit the finished line.
Also added a defensive comment on the SessionLabeler instantiation
documenting that the per-event session-prefix path is currently dormant
in the opencode runtime, but kept in place so attribution flips on
automatically if/when opencode begins streaming subagent sessions.
* fix subagent-finished log: hybrid exact+FIFO callID matching
opencode does not consistently surface a tool_result callID matching the
originating tool_use callID for the `task` tool, so the previous
exact-match-only finish line never fired. Now we:
- Dual-index task dispatches by callID AND in a FIFO queue.
- Track non-task callIDs so we can identify "unrecognised callID" results
as likely-task-with-mismatched-id.
- On tool_result, exact-match first; fall back to FIFO when the output
looks like a subagent reply (>300 chars) and the callID is unknown.
- Flush leftover dispatches at run end with an "(inferred at run-end)"
suffix so the gap is visible if subagent results arrive entirely off
the tool_result event path (e.g. inlined into the next assistant
message).
* fix subagent-finished log: move run-end flush to post-subprocess block
Investigation on T3 + finish-log-validation runs revealed two real issues
with my prior attempt:
1. The `result` event handler is dead — opencode never emits a
`result`-typed event over its NDJSON stream, so the inferred-at-run-end
flush I had placed there never fired. Move the flush to right after
`runSubprocess` returns where it actually executes.
2. The FIFO heuristic was too strict — the >300-char output check
excluded short or empty outputs that opencode's `task` tool_result
appears to carry (the subagent's full reply seems to arrive via a
separate channel, not the result event itself). Drop the size check;
rely solely on `knownNonTaskCallIDs` to keep genuinely-non-task
tool_results from popping a pending task.
Net effect: every `task` tool dispatch gets a matching `» subagent
finished` line in the logs, either from the FIFO fallback during the run
or from the run-end flush as a backstop.
* modes/anneal: anchor lens calibration in worked examples
The prior trivial-skip definition ("single-line fix, formatting-only,
…") was anchored on diff size, but real-world risk is anchored on diff
*shape*: a 5000-line lockfile regen IS trivial, and a 1-line SQL
operator flip in a billing path is NOT. The prior lens-count guidance
("there's no fixed count, bias toward more for high-stakes
subsystems") gave the agent no concrete shapes to anchor against, so
runs varied between under-pick (4 generic lenses on a billing PR) and
over-pick (5 overlapping themed lenses on a refactor).
This commit hardens both:
- Trivial definition gets explicit "looks trivial but isn't"
anti-patterns: SQL operator flips, money/tax/timeout constants,
feature-flag defaults, comparison operator changes, semantic 1-liners
buried in whitespace, public-API renames, new direct deps. Skip lists
get explicit "size doesn't matter" calibration for lockfile regens
and mechanical renames.
- Lens count gets a worked-example ladder: 1 lens (refactor / new test
file / isolated fix), 2-3 lenses (typical features), 4-5 lenses
(high-stakes subsystem touches), 6+ is a smell.
- Subsystem lenses get an explicit recommendation to lead over generic
themed equivalents for high-stakes domains, with the reasoning:
domain framing primes the subagent for domain-specific failure modes
(double-charges, refund races, dispute flows) the generic lens
misses.
Mirrored byte-identical into both anneal.md copies; modes.ts updates
all three review surfaces (Build self-review, Review triage,
IncrementalReview triage).
* fix harness false-failure when Review submits without todowrite
Review and IncrementalReview prompts explicitly forbid calling
report_progress (the review IS the durable record). The post-run
harness in action/utils/run.ts errors with "agent completed without
reporting progress" when toolState.wasUpdated is false at exit. Until
now, the only path that set wasUpdated for these modes was the
todoTracker's debounced publish — which only fires if the agent
happens to call todowrite during the run. Adversarial run on PR #16
(misleading-trivial billing tweak) hit exactly this case: agent went
straight from triage → fan-out → review submission with no todowrite
calls, and the harness reported failure even though the substantive
review was successfully submitted with two inline comments.
Fix: create_pull_request_review now marks wasUpdated=true (and
finalSummaryWritten=true) on every terminal path — successful submit,
empty-content skip, and all-comments-dropped skip. Submitting a review
is unambiguously a "done" signal in these modes.
Found via adversarial testing of the multi-lens orchestrator on a
1-line tax constant change. Logged in /tmp/pullfrog-validation/v3/.
* fix harness false-failure when Review submits without todowrite (correctly)
Replaces the prior fix (acc2bd65) which set wasUpdated=true inside
create_pull_request_review. That approach worked for the harness check
but broke the orphan-comment cleanup: with wasUpdated=true and
finalSummaryWritten=true, the (!wasUpdated || trackerWasLastWriter)
condition in main.ts evaluated false and the "Leaping into action"
progress comment was left behind on every Review run — the exact
behavior the cleanup logic was designed to prevent (see
plans/review_progress_comment_cleanup_b0120f6c.plan.md).
Correct fix: change the harness check in action/utils/run.ts to
recognize a submitted PR review as an alternate completion signal
alongside wasUpdated. wasUpdated stays false on purpose so cleanup
deletes the orphan, but the run no longer false-fails when the agent
followed the Review-mode contract (submit a review, never call
report_progress).
The bug was discovered during adversarial testing of PR #16
(misleading-trivial billing tweak) where the agent went straight from
triage → fan-out → review submission without using todowrite, causing
the harness to error even though the substantive review (a CAUTION
blocking review with two inline comments catching a 10x tax cut) was
successfully posted.
* fix harness false-failure for Review modes (mode-based carve-out)
Replaces the prior carve-out (4c0f69aa) which gated on
toolState.review.id. That worked for runs where the review tool
actually populated the toolState (validation-2 succeeded), but failed
for runs that took a slightly different path where the assignment
didn't propagate visibly to handleAgentResult — even when the review
verifiably posted to GitHub.
Found this empirically: PR #19 (pure mechanical rename across 20
files) opened with the prior fix in place, the agent picked exactly
one impact lens (correct calibration!), confirmed no stale references,
submitted "Reviewed — no issues found." successfully (visible in
GitHub API), and the harness STILL errored with "agent completed
without reporting progress." Same SHA, same branch, same code as
validation-2 which passed. The toolState.review.id check turns out
not to be reliably visible from the run.ts handler in all paths.
Better fix: gate on toolState.selectedMode. Review and
IncrementalReview modes are designed to never call report_progress
(the review is the durable record, and IncrementalReview's
non-substantive path produces no artifact at all by design). The
harness completion check makes no sense for these modes — skip it
entirely. The agent's clean subprocess exit is the completion signal.
This also handles edge cases the previous fix missed: IncrementalReview's
non-substantive path (no review submitted by design) and any future
Review-flow shape that doesn't end at create_pull_request_review.
* ci: trigger Test run to validate models-live timeout/concurrency changes
* ci: prune passthrough models from live smoke matrix
openrouter/* aliases and keyed opencode/* aliases are routing-layer
wrappers around models we already smoke-test directly. running every
passthrough burns CI minutes (~30 min/run) without catching anything
the direct smoke doesn't — slug drift is already covered by the
models-catalog job.
keep one canary per routing layer (openrouter/claude-sonnet,
opencode/claude-sonnet) to validate auth + tool-call translation. free
opencode models stay in the matrix since they're unique to the provider.
INCLUDE_ALL_PASSTHROUGHS=1 bypasses the prune for full validation.
matrix size: 37 → 20 jobs.
* fix isRateLimited false-positive on UUIDs/timestamps containing 429
The bare "429" substring pattern was matching MCP session IDs (e.g.
`...-4429-...`) and microsecond timestamps in agent stdout, sending
transient failures down the 60s rate-limit retry path. With the new
4-minute per-step CI timeout, that backoff plus a slow retry pushed the
step past its budget and timed out.
Switch to regex patterns and gate the numeric code on `\b429\b` so word
boundaries prevent the substring false-match. Verified locally that the
UUID `97287d2f-ae1d-4429-8627-73e2454e80ca` and timestamp `02:04:50.9429654`
no longer match while real `HTTP 429` / `"status":429` strings still do.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* add stop hook + learnings reflection to post-run loop (#515)
stop hook (#515): repo-configured script that runs after the agent
finishes. non-zero exit resumes the agent with the hook output as
guidance; persistent failure (3 attempts) marks the run failed. the
dirty-tree and stop-hook gates share a single retry loop so a fix +
push happen in one turn.
learnings reflection: per Colin, the learnings step baked into mode
checklists rarely fires — the agent stays focused on the task and the
meta-ask falls through. the post-run loop now delivers a dedicated
one-shot --continue turn asking the agent to call update_learnings if
relevant, nothing else competing for attention. reflection doesn't
consume the gate-retry budget; if it dirties the tree, the next loop
iteration catches it via the dirty-tree gate.
plumbing: Repo.stopScript column + migration, zod schema, run-context
api, AgentSettings UI. RepoSettings.stopScript threads through to
AgentRunContext and into each agent harness.
subprocess-dependent logic lives in action/agents/postRun.ts to keep
action/agents/shared.ts lean — shared.ts is reachable from
pullfrog/internal, and pulling node:child_process through it leaks
into root tsc (which uses bundler resolution, not NodeNext).
* fix: preserve successful run when reflection turn fails
The post-run reflection turn (update_learnings nudge) is a best-effort
one-shot; its failure must not flip a successful run to failed. Prior
code overwrote `result` with the reflection's return value, so a model
API error during reflection caused the whole run to be reported as
failed even though the gated work had already completed cleanly.
Now: save the pre-reflection result, and if reflection returns
`success: false`, log a warning, restore the prior success, and exit
without re-invoking the gates (re-running a freshly-green stop hook
risks a flaky false-positive failure).
Adds action/agents/postRun.test.ts covering the reflection path —
previously uncovered.
* fix: surface both stop-hook stdout and stderr to the agent
The `(stderr || stdout)` heuristic in executeStopHook dropped stdout
entirely whenever stderr had any content. Scripts that emit a benign
warning to stderr and the actionable error to stdout (common for
wrapper scripts) starved the agent of the information it needed to
fix the issue.
Now concatenate both streams (stderr first, stdout second, skipping
empty ones) before truncation. This keeps stdout's tail — usually
where summaries and totals live — intact under the 4096-char cap.
* test: lock in the core post-run retry + reflection invariants
PR #548's test plan ships four manual verification scenarios.
Convert three to vitest coverage, catching regressions on the hottest
code paths:
- persistent stop hook failure exhausts MAX_POST_RUN_RETRIES and
surfaces as AgentResult.error with both the retry count and the
verbatim hook output (so the GitHub-comment rendering stays
actionable).
- every gate retry is fed the hook output as the resume prompt.
- usage aggregates across the initial run plus every retry (billing
relies on this).
- reflection turn still fires when no stop hook is configured and the
tree is clean.
Manual item remaining is the full UI round-trip of the settings form,
which is out of scope for unit tests.
* test: cover executeStopHook soft-fail and truncation invariants
Three paths the PR documents but previously had no regression gates:
- timeout (SPAWN_TIMEOUT_CODE) and activity-timeout
(SPAWN_ACTIVITY_TIMEOUT_CODE) must return null, not a failure. a
hook that times out is an infra problem; retrying with an agent
turn risks an infinite loop.
- spawn errors (ENOENT from a typoed binary, etc.) take the same
soft-fail path for the same reason.
- oversize hook output is truncated to the last 4096 chars with a
"truncated" marker, keeping the tail (where summaries live) and
protecting the 65535-char GitHub-comment budget downstream.
Regression targets — a refactor that accidentally surfaces an infra
failure as a gate failure, or blows the comment budget, will now
fail loudly in CI.
* test: cover soft-fail, no-resume, and short-circuit invariants
Three more documented behaviors that previously had no regression
gates:
- dirty-tree-only is a soft-fail: persistent uncommitted changes log
and warn but DO NOT flip the run to failed. a regression that
started surfacing this as AgentResult.error would break every run
that leaves a test fixture untracked.
- canResume=false + stop hook failure still surfaces the hook failure
as AgentResult.error. the retry budget is zero so "N retry
attempts" is correctly omitted from the message, but the run still
reports WHY it failed rather than silently reporting success.
- initial result with success=false short-circuits the loop: no gate
checks, no reflection, no resume calls. the original agent error
flows through verbatim for clean triage.
Also reset mockedSpawn in beforeEach so test state doesn't leak
between cases.
* test: lock in the reflection-dirties-tree → dirty-tree-gate path
The PR description claims: "if the reflection turn dirties the tree,
the loop picks that up on the next iteration via the normal
dirty-tree gate." There was no regression gate on this invariant.
Without it, a refactor that moved the reflection out of the retry
loop (e.g., into a one-shot post-loop call) would silently bypass
the commit-before-you-finish contract whenever the agent misbehaves
during reflection — uncommitted changes would ship as part of the
run's "success" state.
The test sequences three getGitStatus returns (clean → dirty → clean)
and asserts two resume calls: REFLECTION first, then UNCOMMITTED
CHANGES with the dirtying file in the prompt.
* fix: preserve pre-reflection task output when reflection succeeds
the reflection turn's reply ("done" or "updated learnings with N bullets")
is a meta-ask, not a task summary. before this fix, result = reflectionResult
clobbered the original task's output on the returned AgentResult, so
downstream consumers (handleAgentResult's fallback path when toolState is
empty, programmatic callers of main()) saw the reflection's trivial reply
instead of the real summary.
spread reflectionResult to inherit fields subsequent gate retries need
(e.g. the new sessionId claude emits per --resume invocation), but keep
the pre-reflection output verbatim.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: fall back to reflection's output when pre-reflection output is empty
the prior fix used `??` which only fell through on null/undefined. runs
that communicate exclusively through MCP tools (e.g. report_progress) and
emit no plain text leave result.output = "", which `??` preserved as-is —
dropping the reflection's reply and leaving handleAgentResult's fallback
path with nothing to show. switch to `||` so empty-string pre-reflection
output yields the reflection's output instead of ""; non-empty task output
still wins as intended.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test: drop reflection-failure-skips-hook test (over-specified control flow)
the test pinned the literal `break` in the post-reflection failure
branch with stopScript=null, asserting only that getGitStatus was
called once. that's not a behavior contract — a reasonable refactor
(e.g. `continue` to re-check gates with explicit flake guards) would
fail this test even though the new behavior would be fine. the
"does not flip a successful run to failed" test already covers the
only thing callers depend on.
* test: drop low-value mock-driven tests from postRun
- "fires the reflection turn when no stop hook is configured" — fully
subsumed by the output-preservation test (asserts task output
survives, which is only possible if reflection fired).
- "uses stdout alone" / "uses stderr alone" — pin format trivia
(`filter(Boolean).join`) that LLMs ignore.
- "returns empty output (not undefined) when both streams are empty"
— guards a TS-impossible case; every consumer uses `output || "(no output)"`.
- "returns null on activity-timeout" — duplicate of the timeout test;
same `return null` branch with a different constant.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* fix(#15): precompute diff anchors in checkout_pr TOC
* test(#15): update TOC snapshot for precomputed diff anchors
* chore(tests): skip codex-mini-latest models.dev check + refresh latest-by-provider snapshot
* fix(#22): add commitCount and commitLog to checkout_pr return
* fix(#21): include PR body in checkout_pr return
* fix(#5): force-fetch PR refspec to overwrite stale local branch
* fix(#31): rename git tool parameter from subcommand to command
* fix(#11): soft-fail post-checkout hook, bump timeout to 10min
* fix(#16): strengthen diff file usage guidance
Agent was bypassing diffPath and running `git diff` instead. Tighten
instructions in `checkout_pr` result and remove the mixed-signal
"log, diff" listing in the global Git guidance. `git log` and
`git diff --stat` remain allowed for commit-range overview.
* fix(#20): drop invalid inline review comments instead of failing review
Previously, a single inline comment anchored outside a diff hunk would
422 the entire review submission. Pre-validate comments against the
PR file patches via listFiles, drop the invalid ones, and append a
note to the review body listing what was skipped. Include the dropped
list in the tool response so the agent can retry targeted fixes.
* fix(#12): stop MCP server on inner activity kill + filter reconnect noise
Inner-activity-kill zombies were burning multi-hour runner time because
mcp-proxy's SSE reconnect and provider-error retry lines kept the outer
activity timer alive long after the agent subprocess was killed.
- Filter [mcp-proxy] / "provider error detected" chunks so they don't
count as outer-timer activity.
- Add onActivityTimeout callback to spawn + thread through agent runs.
- main.ts wires that callback to stop the MCP HTTP server (so reconnects
finally fail instead of looping) and arms a 5min safety-net timer that
force-rejects the outer timer if the agent promise is still pending.
* audit: harden #12 lifecycle + cover #20/#12 with unit tests
Bugs found during Ralph audit of the prior run-issues fixes:
- main.ts's 5min safety-net setTimeout was never cleared on the happy
path; also activityTimeout.stop() didn't null the internal rejectFn,
so a late forceReject from the safety-net could still reject a
long-resolved promise. Timer now cleared in finally; stop() now
disarms forceReject.
- mcp server disposal was non-idempotent, so the inner-kill path ran
server.stop() twice once the outer `await using` block exited. Made
the returned disposer idempotent.
Tests:
- action/mcp/review.test.ts: 14 tests for commentableLinesForFile
(multi-hunk, no-count hunks, no-newline marker, empty) and
validateInlineComments (file not in diff, wrong side, out-of-range
line and start_line, partitioning batches, default side).
- action/utils/activity.test.ts: 6 tests for isActivityNoise covering
mcp-proxy lines, provider-error lines, mixed chunks, Buffer input.
* audit(#22): cap commitLog at 200 + scope git-diff restriction to PR review
- cap git log --oneline at 200 entries so a PR with thousands of commits
cannot blow up the MCP tool response; expose commitLogTruncated so
callers can warn the agent when the log was clipped
- tighten instruction wording so `git diff` / `git diff --cached` remain
available for inspecting an agent's own uncommitted changes, while
PR review content must still come from diffPath
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#11,#22,#31): surface hook/commit warnings in instructions + polish git tool
- append hookWarning + commitLogTruncated advisories to checkout_pr
instructions so the agent actually sees the warning inline, not just
as a field it may skip
- fix stale 'subcommand' wording in git tool redirect for `pull` and
in the `command` parameter description; the MCP parameter is named
`command` now, and that's what the agent binds to
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(#20): reassign params.comments even when all inline comments dropped
if every inline comment fails pre-validation, the earlier guard skipped
reassigning params.comments, so the submission still carried the bad
comments and GitHub 422'd on the whole review. always reassign to
validation.valid so the downstream 'nothing left to post' skip fires
and an otherwise-empty review is no-oped cleanly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#22): degrade gracefully when base ref isn't resolvable
checkout_pr used to assume \`origin/<base>\` is always reachable, but
it isn't guaranteed after a shallow fetch that only pulled down the PR
head. Failing the whole checkout over metadata we added for ergonomics
would be a regression, so wrap the rev-list / log in a try/catch and
return empty commit metadata instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#12): anchor noise patterns to line start to avoid false positives
before this, a line like "agent said: [mcp-proxy] was there" or
"context: provider error detected in log" in real agent output would
have been treated as noise and failed to reset the outer activity
timer. both patterns now anchor at the start of the (optionally
debug-timestamped) line, matching only lines mcp-proxy or our own
log.info actually emit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#20): export and unit-test formatDroppedCommentsNote
covers single-line `path:N`, multi-line `path:start-end`, and
startLine==line fallback so changes to the dropped-comments note
format surface in test diffs instead of only in GitHub UI.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#20): cap dropped-comment note to stay under GitHub body limit
a pathological run (agent emits hundreds of invalid inline comments
on a huge PR and they all get dropped) would push the review body
past GitHub's ~65KB limit and fail the whole submission with a
body-too-long 422 — the exact all-or-nothing failure #20 was meant
to prevent. cap the detail list at 50 entries with a "…and N more"
line so the note stays bounded.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#20): distinguish binary/no-patch files in dropped-comment reason
previously a comment on a binary file (or pure rename / mode-only
change) was dropped with "line X is not inside a diff hunk", which
misleads the agent into retrying with different line numbers. call
out the no-textual-diff case explicitly so the agent knows to move
that feedback to the review body instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#11): replace lifecycle timeout string-match with typed sentinel
spawn() now rejects with SpawnTimeoutError (code === SPAWN_TIMEOUT_CODE or
SPAWN_ACTIVITY_TIMEOUT_CODE) instead of a plain Error. executeLifecycleHook
now branches on that code so rewording the error message in subprocess.ts
can no longer silently misroute timeouts into the "transient — retry"
warning.
* audit(#12): route agent hung-vs-failed via typed SpawnTimeoutError
claude.ts and opentoad.ts decide between "hung" and "failed" log wording
based on the subprocess error. move them off the literal "activity
timeout" substring match onto the same SPAWN_ACTIVITY_TIMEOUT_CODE
sentinel used by lifecycle.ts so all three call sites agree on the
source of truth.
* audit(#20): delete leftover pending review when submit fails
Why: `createAndSubmitWithFooter` creates a PENDING review first so we can
mint Fix-links with the review ID, then submits. If submitReview fails
(e.g. 422 from a race where the diff moved between pre-validation and
submission), the draft was left on the PR. GitHub only allows one
pending review per user, so the agent's retry would then fail with
"already has a pending review" — an error the agent has no tools to
clean up from.
Best-effort cleanup: delete the pending draft on submit failure before
re-throwing the original error, so retries start from a clean slate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#31): point agent to concrete alternative when rebase/bisect blocked
Why: in disabled-shell mode, `git rebase` and `git bisect` are blocked as
arbitrary-code-execution escape hatches. Previous error messages
explained *why* but left the agent without a next step — especially
painful right after the `pull` redirect, which suggested "merge or
rebase locally." The agent would follow that advice, hit the rebase
block, and loop without knowing what to try next.
Now: rebase block explicitly says "use 'merge' instead"; bisect block
notes that manual bisect is also unavailable through this tool; pull
redirect no longer recommends rebase in shell-disabled contexts.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: import security tables into security.test to prevent drift
Why: the security tests re-declared AUTH_REQUIRED_REDIRECT,
NOSHELL_BLOCKED_SUBCOMMANDS, and NOSHELL_BLOCKED_ARGS inline with
hand-copied message strings. When the runtime messages in git.ts were
tightened (recent rebase/bisect guidance updates), the test copies
drifted and tests validated a stale version of the logic while passing
clean. A missing or mistyped entry in git.ts could therefore slip
through.
Now: export the tables from git.ts and import them into the test file.
If a runtime message changes, the tests exercise the new string
automatically; if an entry is added or removed, tests covering that
command see the change without manual sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: widen pending-review cleanup to cover pre-submit throws
getApiUrl() (invoked in footer build) can throw if API_URL is
misconfigured, which would leak a pending draft between createReview
and the previous submitReview try/catch. Move the try/catch to wrap
the entire post-create body so any throw routes through
deletePendingReview cleanup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: reject leading-dash refs/branch names to block flag injection
git's parseopt accepts options intermixed with positional args, so a ref
like "--upload-pack=evil" passed to git_fetch could be parsed as a flag
rather than a refspec. Add a narrow rejectIfLeadingDash helper to
git_fetch (ref), delete_branch (branchName), and push_branch
(branchName). HTTPS remotes ignore --upload-pack server-side, but the
hygiene matters for defense in depth (ssh remotes, future code paths).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: validate the resolved branch in push_branch too
When branchName is omitted, rev-parse surfaces the current branch name,
which could start with '-' if git state was tampered with. Move the
leading-dash check to after the branch is resolved so both the explicit
and derived paths go through validation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: cache commentable-lines snapshot at checkout to match review anchor
Review comments are anchored to checkoutSha (commit_id), but validation
was hitting pulls.listFiles at review time — latest HEAD, not the SHA the
agent actually reviewed. If the PR was updated mid-run, valid comments
could be silently dropped (or invalid ones admitted). Snapshot the
commentable lines during checkout_pr so review-time validation matches
the anchor exactly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#12): route activity monitor's own debug output around the write wrap
startProcessOutputMonitor monkey-patches process.stdout.write to mark
activity, then called log.debug(...) every 5s to report idle time — which
landed right back in its own wrapper, failed isActivityNoise, and called
markActivity. with ACTIONS_STEP_DEBUG=true (common on reruns) the idle
counter reset every interval and the timeout could never fire,
re-creating the #12 zombie-run bug for any debug-enabled run.
Fix: capture the original stdout.write and use it directly for the
monitor's own diagnostics so they bypass the feedback loop. Added a
tight-timeout regression test that asserts the timeout still rejects in
debug mode.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#12): noise-filter subprocess.ts monitor logs so outer timer survives debug
activity.ts's own monitor output already bypasses the wrap (c35cd3fb),
but subprocess.ts's spawn activity timer uses log.debug — which goes
straight through process.stdout.write and would still mark activity on
every interval when debug logging is enabled. Pattern-filter those
'(spawn|process) activity (check|timer|monitor)' lines in both local
([DEBUG] ...) and GH-runner (::debug::...) formats so they don't reset
the outer agent-hang timer.
Kept scoped to those specific monitor messages — a blanket [DEBUG]
filter would silently classify any coincidentally-debug-prefixed agent
output as idle, which is a worse failure mode than the one we're
fixing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#11): surface spawn ENOENT-style errors in stderr buffer
spawn() resolved with exitCode=1 and an empty stderr when the command
itself couldn't start (missing binary, bad permissions). lifecycle.ts
then reported 'output: (empty)' to the user, who was explicitly told
'retry if the failure looks flaky' — so every run hit the same wall with
no diagnostic trail.
Append the '[spawn] <cmd>: <node error>' line to stderrBuffer before
resolving so the real cause (ENOENT, EACCES, …) flows through to the
hook-warning message.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit(#11,#12): cover executeLifecycleHook typed-timeout routing
the typed SpawnTimeoutError + sentinel-code branching introduced in
d7ee7fd2 / ea8dd2c4 classifies hung vs failed lifecycle hooks — critical
for whether agents retry — but had no unit coverage. add tests for all
four branches (no script, exit 0, non-zero exit with retry-if-flaky
guidance, timeout with do-NOT-retry guidance, transient spawn failure).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: re-verify clean tree after prepush hook
the pre-prepush check guarantees we enter the hook with a clean tree, but
if the hook writes tracked files (formatter, type generator, build
artifacts), the push still only sends the pre-hook commit — the hook's
edits silently disappear from the upstream branch while the tool reports
"successfully pushed". add a post-hook status check so the agent sees the
dropped mutations and can commit or discard them before retrying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: reject push_tags refspec injection via ':' in tag name
without tag validation, a tag like "foo:refs/heads/main" concatenated into
"refs/tags/${tag}" becomes a valid <src>:<dst> refspec — git pushes the
local refs/tags/foo's commit to remote main, bypassing push_branch's
default-branch guard. same shape blocks leading '-' (flag injection) and
other refspec metacharacters (~ ^ ? * [ \) via an allow-list regex.
only reachable in push=enabled today, so this is defense-in-depth, but
hardens the tool in case push_tags is ever exposed in restricted mode.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: stop pointing agents at an internal constant they can't change
the lifecycle-hook timeout warning told agents to "bump
LIFECYCLE_HOOK_TIMEOUT_MS" — but that's a hard-coded constant in the
action, not something the agent or repo owner can tune. the agent would
plausibly loop hunting for where to change it. redirect to the actual
lever they control: ask the repo owner to simplify the hook.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: drop inverted inline-comment ranges locally with precise reason
validateInlineComments only checked that both line and start_line anchor
inside a hunk, not that start_line <= line. an inverted range (e.g.
start=44, line=42) would pass local validation and GitHub would 422 with
"invalid line numbers" — opaque to the agent and unfixable without
reading docs. reject locally with a reason that names the constraint.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: don't let usage-summary write error mask main's outcome
writeGitHubUsageSummaryToFile is called in main's finally block. it can
throw on ENOSPC / EACCES / missing parent dir. a throw here propagates
past the try's successful return or the catch's error return, hiding the
actual run outcome behind an I/O failure on a purely informational file.
swallow the write error (debug-logged) — the summary is nice-to-have, not
load-bearing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: don't mislabel agent handler errors as JSON parse failures
the onStdout event loop wrapped both JSON.parse and the handler call in
one try/catch that logged every caught error as 'non-JSON stdout line'.
if a handler threw (e.g. todowrite state shape drift), the error was
silently classified as a parse error, making diagnosis impossible. split
the try blocks so JSON errors and handler errors get distinct,
identifying log lines.
* audit: reject leading-dash PR refs before they reach git commands
PR head/base refs come from GitHub and are attacker-controlled on fork
PRs (the PR author picks headRef freely). they flow straight into
`git fetch origin <ref>`, `git checkout -B <ref>`, and config writes.
without a leading-dash check, a ref named like '-upload-pack=evil'
could be parsed as a flag instead of a refspec.
validate both refs at the top of checkoutPrBranch (before any async
work) and cover the two attack shapes with unit tests.
* audit: cover ActivityTimeout.stop()'s forceReject disarming
main.ts's safety-net-timer path depends on ActivityTimeout.stop()
nulling out rejectFn so a late safety-net fire after a successful
agent run is a no-op. that behavior had no direct coverage — removing
the \`rejectFn = null\` in stop() would silently break the happy path
(unhandled rejection / spurious failure) without failing any test.
add three tests covering: forceReject rejects with the reason,
stop() disarms forceReject, and forceReject after timer rejection
is an idempotent no-op.
* audit: stabilize activity-timeout idleSec against late stdout race
* audit: reject 0ms timeout parses to avoid insta-fail from '0m'
* audit: surface raw GitHub error on review 422 instead of assuming anchor cause
* audit: key commentable-lines cache by PR number to prevent cross-PR drift
* audit: enumerate concrete 422 causes and name checkout_pr in review error
* audit: stop shipping ralph-loop runtime state in PR history
.claude/ralph-loop.local.md and .claude/ralph-loop-prompt.md were
accidentally staged in an earlier audit commit. the .local.md suffix is
conventional for gitignored runtime state, and the prompt file is
per-run harness config — neither should merge to main. ignore the
pattern and untrack the existing entries (files remain on disk so the
active loop keeps working).
* audit: pin commentable-lines cache to checkoutSha, not just PR number
a second checkout_pr(N) call advances toolState.checkoutSha at line 305
or 334, then runs fetchAndFormatPrDiff + cache population at line 549.
any throw between those two points (rate limit, 5xx, network blip) left
the old snapshot keyed to (pullNumber=N) while checkoutSha now points at
a different sha. review_pr(N) would reuse the stale snapshot, silently
validating comments against the wrong anchor — the original failure this
cache was meant to prevent.
track commentableLinesCheckoutSha alongside the pull number and require
both to match before returning the cache. if either has moved, fall
back to listFiles like any other miss.
* audit: auto-clear leftover pending review from killed prior runs
a workflow timeout or OOM between createReview PENDING and submitReview
leaves GitHub holding a pending draft. the next run hits GitHub's
one-pending-per-user-per-PR limit and 422s at pending-create, with no
way to recover short of a human cleaning up manually.
catch 422 at pending-create, list the PR's reviews (GitHub only exposes
our own pending to us, so the filter is safe), delete the leftover, and
retry once. 404/422 on the cleanup are treated as no-ops (race with
another concurrent cleanup or the draft was submitted); any other
cleanup error rethrows so the real cause reaches the caller.
* audit: extract + unit-test stranded-pending-review cleanup
the recovery branch inside createAndSubmitWithFooter had no direct test
coverage. a regression in any of its guards (status check, message
match, listReviews filter, 404/422 tolerance, non-retryable rethrow)
would silently cause either destructive deletes of unrelated reviews or
the old failure mode where a stranded pending draft blocks every retry.
extract to clearStrandedPendingReview so the cases can be exercised with
a mocked octokit, and add tests for each branch — including the
load-bearing negative cases (non-422 passthrough, non-pending-review 422
passthrough, no-leftover-found passthrough, non-retryable cleanup error
passthrough). no behavior change at the call site.
* audit: document concurrent-run race in clearStrandedPendingReview
two runs on the same PR using the same GitHub App installation token would
both see each other's PENDING draft via listReviews (GitHub exposes PENDING
only to the author, and both runs share authorship). the loser's recovery
path would delete the winner's active draft, causing the winner's
submitReview to 404.
no reliable in-request signal distinguishes a genuinely-stranded prior-run
draft from an active peer's draft — PENDING reviews have no created_at,
and the user field is the same bot in both cases. the correct fix is
workflow-level concurrency (a per-PR concurrency key), not a heuristic
here. document the limitation so future readers don't try to bolt on a
broken heuristic.
* audit: report signal-killed subprocesses as failures, not exit code 0
node's close event delivers (code=null, signal=<name>) when a child is
killed by signal (OOM killer, segfault, external SIGTERM). the close
handler captured only exitCode and coerced null to 0 via `exitCode || 0`,
so lifecycle hooks killed by signal were silently reported as successful —
lifecycle.ts's `if (result.exitCode !== 0)` check skipped the warning and
callers proceeded as if setup/post-checkout/prepush had completed.
now capture signal, append "killed by signal <name>" to stderr, and
resolve with exitCode=1 when code is null but signal is set. adds a
regression test that spawns `kill -KILL \$\$` and asserts a non-zero
exit plus the signal-kill marker in stderr.
* audit: untrack RUN_ISSUES*.md ralph-loop working docs
same pattern called out in 4f14dbf1: these files are per-run harness
state and analysis scratch, not merge-to-main deliverables. the TODO
literally opens with "Ralph loop instructions:", so it's unambiguously
in the same category as .claude/ralph-loop-prompt.md was. files stay on
disk so the active loop keeps working.
* audit: block refs/... + symbolic-ref bypass of default-branch guard
push_branch's restricted-mode guard compared the resolved remoteBranch
against defaultBranch with exact-string equality. an agent passing
branchName "refs/heads/main" flowed through: rejectIfLeadingDash passed,
getPushDestination's fallback preserved the refs/heads/main string as
remoteBranch, so "refs/heads/main" !== "main" and the block was skipped,
yet git push happily resolved refs/heads/main to the local main commit
and pushed to the remote main branch. symbolic refs (HEAD / FETCH_HEAD /
ORIG_HEAD / MERGE_HEAD) are the same class of bypass — they resolve to
whatever commit they point at, unconstrained by the name-based guard.
add rejectSpecialRef to enforce bare branch names at the tool entry, use
it in push_branch and delete_branch. checkout_pr only ever assigns
pr-<number> as the local branch, so nothing legitimate relied on the
refs/... form here.
* audit: keep original 422 visible when listReviews fails during pending-review cleanup
if listReviews threw (e.g. transient 502, rate limit) during the stranded
pending-review recovery path, the listing failure replaced the original
422 "pending review" error when it propagated up through the tool's outer
catch. agents then saw a generic server error with no mention of the real
blocker and stopped retrying the cleanup.
now the listing failure is logged at debug but does not mask the original
422. the caller's retry re-attempts cleanup, which succeeds if the listing
failure was transient.
* audit: block default-branch deletion even under push: enabled
delete_branch required push: enabled, but within that mode the agent
could delete the default branch with no local guard. GitHub branch
protection usually catches this at the remote, but not every repo
has protection configured — and even when it does, relying on remote
config for local safety is wrong. pushing to main is reversible
(revert, force-push old HEAD); deleting main is not (reflog recovery
only, 30-day window).
block deletion of the resolved default_branch in DeleteBranchTool
regardless of push permission. push: enabled authorizes pushes, not
wholesale removal of the repository's primary branch.
* audit: attach no-op catch to agentPromise so a late rejection can't crash cleanup
agentPromise raced against activityTimeout.promise (and the --timeout
timeoutPromise), both of which had .catch(() => {}) handlers. agentPromise
did not. if a timeout won the race, agentPromise became stranded and its
subsequent rejection was an unhandled rejection — under node 15+'s default
unhandled-rejection policy that terminates the process, which would kill
main() mid-cleanup and lose the error-reporting and usage-summary work
queued in the catch/finally blocks.
the race still sees the rejection (the original promise is shared); this
catch only prevents node from treating a post-race rejection as unobserved.
* audit: close push_branch refspec-injection via ':' / '+' in branchName
rejectSpecialRef only forbade leading-dash, `refs/` prefix, and symbolic
refs. git push accepts `[+]src[:dst]` refspec syntax, so an agent under
push:restricted could smuggle a full refspec through branchName and bypass
the downstream exact-string default-branch guard:
"evil:refs/heads/main" → push local 'evil' to remote main
":refs/heads/main" → delete remote main
":other" → delete arbitrary branches (outside grant)
"+main" → force-push refspec prefix
reject ':', '+', '^', '~', '?', '*', '[', '\\', and whitespace — git's own
check-ref-format forbids all of them in branch names, so the allow-list
cannot false-positive against a legitimate branch. add regression tests.
* audit: stop suggesting blocked 'rebase' in push_rejected advice under shell=disabled
Why: when push fails with non-fast-forward, the advice told the agent to run 'git rebase origin/...'. In shell=disabled mode the git MCP tool blocks rebase (as an arbitrary-code-execution escape hatch), so the agent's only path forward was to hit the block, read the fallback message, and try merge — one wasted round trip.
Now: under shell=disabled we directly suggest 'git merge origin/...', which always works. Under other modes the advice keeps the rebase/merge choice but leads with merge so the example is copy-pastable either way.
* audit: harden includeIf cleanup against shell-injection via subsection names
setupGit read `includeif.*` keys via `git config --get-regexp`, split on the
first space, and fed the result into `execSync(\`git config --unset
"${key}"\`)`. git config subsection values preserve arbitrary characters,
so a crafted `[includeIf "gitdir:$(touch${IFS}/tmp/pwn)safe"]` entry
round-trips through `--get-regexp` with its `$(...)` command substitution
intact, survives the split-on-space filter (IFS-bypass leaves the payload
space-free), and gets evaluated when interpolated into the shell command.
Confirmed reachable as an RCE sink in local repro.
Switch to `--get-regexp -z` (null-terminated, no ambiguity on whitespace)
and call `$("git", ["config", "--unset-all", key])` which uses spawn-array
and never hands the key to a shell. Extract the logic into
`removeIncludeIfEntries` and add regression tests covering the injection
payload, whitespace-in-subsection keys, benign entries, and the no-op case.
* audit: clear SIGKILL escalator on clean SIGTERM exit
the overall-timeout path scheduled a 5s SIGKILL follow-up without
capturing the timer id. if the child cooperated with SIGTERM and
`close` fired promptly, the escalator stayed pending in the event
loop for up to 5s — delaying any subsequent clean shutdown (e.g.
the main action exiting after an agent timeout) by that long.
capture sigkillEscalatorId alongside timeoutId and clear it in both
close and error handlers. regression test asserts the active-timer
count does not grow past the pre-spawn baseline after a timed-out
child exits on SIGTERM.
* audit: correct rebase-availability hints to reflect shell=restricted
the MCP git tool only blocks rebase when shell=disabled
(NOSHELL_BLOCKED_SUBCOMMANDS check in GitTool). under
shell=restricted, git({command: "rebase"}) works fine through the
tool — NOSHELL_BLOCKED_SUBCOMMANDS doesn't apply. but two
agent-facing messages implied rebase is only available with
shell=enabled:
- AUTH_REQUIRED_REDIRECT["pull"] said "rebase is only available
when shell is enabled"
- push-rejected integrateStep (non-disabled branch) said
"(or 'rebase' if shell is enabled)"
under shell=restricted, agents reading these would wrongly think
they had to pick merge — pushing them toward merge commits when
rebase would have been cleaner. the push-rejected branch is
already ternary-gated on shell !== "disabled", so the qualifier
there was just redundant noise.
* audit: block difftool/mergetool under shell=disabled
git difftool -x <cmd> is the short form of --extcmd. the args
blocklist only matches --extcmd / --extcmd=*, so -x slipped
through and let an agent run arbitrary commands even when
shell=disabled. globally blocking -x would false-positive on
git cherry-pick -x, which only appends metadata, so block
difftool (and mergetool, same shape via mergetool.<name>.cmd)
at the subcommand level instead. agents have no legitimate need
for either — diffs go through diff/show and merges are resolved
by file edits.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* audit: recover stranded PENDING drafts on no-body createReview too
The body path already clears a stranded PENDING draft from a prior
crashed run via createAndSubmitWithFooter's own try/catch. The no-body
path (approve-with-no-feedback or comments-only) called createReview
directly — so a PR whose previous body-path run crashed between
createReview(PENDING) and submitReview would permanently 422 any
subsequent no-body review with "already has a pending review" until a
body-path run happened to clear it.
Factored out createReviewWithStrandedRecovery so both paths get the
same recovery treatment, and added regression tests covering the
no-stranded / stranded-and-retry / non-stranded-422-no-retry cases.
* audit: reject timeouts past node's setTimeout ceiling
a user-supplied timeout like "999h" parses fine (parseTimeString has no
upper cap) but falls off the 2^31-1 ms limit setTimeout clamps to 1ms.
the agent run would reject with "timed out after 999h" in a single tick.
extract a resolveTimeoutMs helper that centralizes the zero/overflow/
unparseable checks (previously scattered behind inline boolean logic in
main.ts) and cover the behavior with unit tests including the boundary
value.
* fix(#22): replace parameter property in SpawnTimeoutError
node --experimental-strip-types rejects readonly/public/private param
properties in constructors. tests run via node directly (no tsc), so CI
was hitting ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX on every action-agents /
action-agnostic job before any test code ran.
declare the field and assign in the body instead.
* audit: tighten git tool description and delete_branch refspec
- `git` tool description previously implied `pull` had a dedicated MCP tool
alongside `push_branch`/`git_fetch`. it doesn't — the redirect sends the
agent back to the same git tool with `command: "merge"` (or `rebase`).
update the description to teach this directly instead of letting agents
discover it through the redirect error.
- `delete_branch` now passes `refs/heads/${branchName}` to `git push --delete`
so a same-named tag can't be silently deleted when both exist on the
remote. `rejectSpecialRef` already guarantees the bare-name invariant, so
the template construction stays injection-safe.
Made-with: Cursor
* audit: polish review.ts per anneal findings
- drop `as "LEFT" | "RIGHT"` cast in `validateInlineComments` — octokit
types `side?: string` at the createReview endpoint, so narrow via
`c.side === "LEFT" ? "LEFT" : "RIGHT"`. no cast, no redundant
annotation — TS infers the literal union from the ternary.
- consolidate `clearStrandedPendingReview` from 3 params to 2 by folding
`originalErr` into `params`, per AGENTS.md "max 2 parameters" rule.
updates both call sites (`createReviewWithStrandedRecovery`,
`createAndSubmitWithFooter`) and all 7 test paths.
- upgrade `listReviews`-during-cleanup failure log from `log.debug` to
`log.info` so operators not running at debug still see that recovery
was attempted before the original 422 bubbles up. message now reads
"surfacing original 422" to make the intent unambiguous.
Made-with: Cursor
* audit: signal partial commit metadata in checkout_pr
previously a rev-list/log failure (e.g. shallow fetch where
`origin/<base>` isn't reachable) silently returned `commitCount: 0,
commitLog: ""` — indistinguishable from "this PR has no commits past
base", which could mislead review reasoning about scope.
add a `commitLogUnavailable: boolean` field to `CheckoutPrResult`, set
when the rev-list/log calls throw. instructions footer now tells the
agent to treat the values as "unknown" rather than "no commits" in that
case. message phrased to cover the rare case where rev-list succeeds
but git log throws (partial, not strictly zero) metadata.
Made-with: Cursor
* audit: fix parseDiffTocEntries to match production ' · diff-<sha>' TOC suffix
the regex required $ right after the line range, but formatFilesWithLineNumbers
in checkout.ts appends ` · diff-<sha256>` so agents have the GitHub "Files Changed"
anchor precomputed. result: tocEntries was always empty on real PR reviews,
breakdown.files was empty, and runDiffCoveragePreflight never fired its
one-time "read the diff" nudge. add an optional suffix to the regex and a
regression test that uses the exact production TOC shape.
Made-with: Cursor
* audit(#20): skip empty downgraded-APPROVE reviews before they 422
GitHub rejects `event: "COMMENT"` reviews with no body and no inline
comments (HTTP 422 "Unprocessable Entity", verified empirically on
repos/pullfrog/preview-546-run-issues-fixes/pulls/1). the runtime
`prApproveEnabled` downgrade folds approved=true into event=COMMENT
when the repo flag is off, so an agent asking to APPROVE a PR with no
other feedback produces exactly that rejected shape — but the existing
empty-review skip only fired for !approved cases, so the tool POSTed
the doomed COMMENT, octokit returned what looked like a success-with-
no-persisted-review shape, and agents reported a phantom reviewId that
404s on any subsequent GET.
extract the skip decision into `reviewSkipDecision` and add a second
branch for approved + !prApproveEnabled + empty. the function returns
null when the review should be submitted, so a real bare APPROVE
(approved + prApproveEnabled + empty) still goes through unchanged —
GitHub accepts empty APPROVE reviews because the stamp itself is the
content.
surfaced in the PR #546 preview e2e run 24678139563 (reviewId
4141786854 reported by the agent but absent from every reviews
listing). TC13 run 24680349445 re-ran the same scenario with
prApproveEnabled=enabled and the review persisted correctly, isolating
the cause to the downgrade + empty interaction.
* audit(#31): drop misleading rebase mention from pull redirect
AUTH_REQUIRED_REDIRECT["pull"] and the git tool's top-level description
both said "use git_fetch then this tool with command 'merge' (or
'rebase' unless shell is disabled)". the "(or 'rebase' unless shell is
disabled)" qualifier is active misinformation when the agent is
already running under shell=disabled: rebase is blocked there by
NOSHELL_BLOCKED_SUBCOMMANDS, so the suggestion sends the agent into a
second block on the next tool call.
3b83ee97 already fixed this pattern for the push-rejected advice at
line 248, but the pull redirect at line 280 and the tool description
at line 351 were missed. the right copy isn't a conditional qualifier
that agents have to parse against their own shell mode — it's just
naming the one alternative that works everywhere (merge). agents under
shell=restricted/enabled who want rebase can invoke it directly; the
redirect doesn't need to advertise it.
verified in preview e2e run 24679728733 (TC8 probe 6) where the agent
correctly captured the verbatim redirect message under shell=disabled
and explicitly flagged the "(or 'rebase' unless shell is disabled)"
clause as confusing — the new test in security.test.ts asserts the
message names merge and never rebase in every shell mode.
* audit: drop vestigial entry/post references + add preview-546 settings util
followup to d79860c6 "refactor: flatten action entrypoints" (Apr 10),
which moved action.yml from built `entry`/`post` files to source
`entry.ts`/`post.ts` but left three stale references lying around:
- .gitignore: `action/run/entry` / `action/dispatch/entry` paths no
longer exist anywhere in the build.
- .github/workflows/pull-from-action.yml: agent instruction told the
upstream sync agent to "Ignore `entry` files (they are built artifacts
and .gitignored in this repo)". there are no built entry artifacts
anymore — entry.ts is source.
- .cursor/settings.json: search.exclude pattern "**/entry" excluded the
old built files that no longer exist.
none of these were load-bearing on their own, but the same drift had
already broken preview e2e end-to-end: the pullfrog/template workflow's
three-file copy step (cp .../entry, cp .../post) silently failed with
cp: no such file on every preview PR since Apr 10. that template fix
went to pullfrog/template@7ec7c8d and the preview-546 mirror at
@17ab585, which is what unblocked this PR's full e2e validation.
also adds scripts/preview-546-settings.ts, the helper used during the
e2e validation to show/set/reset DB-level repo settings on the Neon
preview branch (push, shell, prApproveEnabled, hook scripts). scoped
to this preview repo ID so it can't accidentally mutate prod.
* audit(#11): scope removeIncludeIfEntries to repoDir under inherited GIT_*
the function takes `repoDir` as the target, but plain execSync / $(...)
inherit GIT_DIR, GIT_WORK_TREE, and GIT_INDEX_FILE from the parent
process — and `git config --local` honors GIT_DIR over cwd. when this
runs as a child of another git invocation (notably the pre-push hook,
but also any future caller embedded inside a git subcommand), the
cleanup silently targets the outer repo instead of repoDir. latent
today because the real caller is ASKPASS setup, which runs before any
git-subcommand ancestor exists, but the function's contract still
promised the wrong thing — and the test suite hit exactly this bug
when invoked through `git push`.
- envScopedToRepo() strips GIT_* before both the get-regexp and unset
calls, so cwd wins.
- swap the $(...) shell helper for execFileSync on the unset call. $()
would merge our scoped env with a "restricted" base that's tuned for
hook execution (no tokens) — overkill here and it re-introduces the
shell-vs-argv distinction this function was explicitly hardened
against in a9aa3b2b. execFileSync with argv is the right tool for a
call where the key can contain arbitrary characters.
- setup.test.ts also strips GIT_* in its own execSync harness so the
suite passes identically under `pnpm vitest run`, `pnpm -r test`,
and `git push`'s pre-push hook.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* fix false "without reporting progress" error + live todo tracking
clean up orphaned progress comments when review is skipped or only
set_output is used, preventing the false positive in handleAgentResult.
parse todowrite events from OpenCode's NDJSON stream and render a
live markdown checklist in the PR progress comment (2s debounce).
agent's explicit report_progress always takes priority.
Made-with: Cursor
* fix contradictory review/progress prompting
align Review and IncrementalReview mode prompts with their guidance —
mode prompts said "always submit" while guidance said "skip if clean."
remove the empty-approval submission that was silently dropped by the
tool. make progress comment lifecycle explicit: created on first call,
updated in place, removed after review submission.
Made-with: Cursor
* centralize todo tracking into shared TodoTracker module
extract inline todo tracking logic (~95 lines) from opentoad.ts into
action/utils/todoTracking.ts. the tracker is created once in main.ts
and passed to agents via AgentRunContext.todoTracker, making it
agent-agnostic and reusable for future agent implementations.
Made-with: Cursor
* fix todoTracker optional type to match file convention
add | undefined to todoTracker in AgentRunContext, matching every
other optional property in the same interface.
Made-with: Cursor
* instruct agents to always maintain a task list for live progress
system prompt now tells agents to create an internal task list at
the start of every run. the tracker renders it to the progress
comment automatically. report_progress is reserved for final
results only — no more intermediate "Checking..." messages that
cancel the tracker and leave stale text on the comment.
Made-with: Cursor
* require report_progress summary at end of every run
agents must always call report_progress with a final summary —
the completed task list should never be the end state of the
progress comment. updated all review mode prompts to call
report_progress after submitting (or not submitting) a review.
Made-with: Cursor
* keep progress comment after review with final summary
stop deleting the progress comment after review submission —
the agent now always calls report_progress with a summary at
the end, and that summary should persist as a record of what
was done.
Made-with: Cursor
* harden stranded progress comment cleanup
- main.ts: detect when tracker was last writer (agent never called
report_progress) and delete the stranded checklist instead of
leaving it as the final comment state
- postCleanup.ts: expand stuck-comment detection to also catch
stranded todo checklists (regex match for checklist patterns)
when the process is killed before normal cleanup runs
- modes.ts + selectMode.ts: add report_progress step to Summarize
and SummaryUpdate modes (only modes that were missing it)
Made-with: Cursor
* fix stale comments, typo, and build mode redundancy
- comment.ts: update deleteProgressComment docstring and inline comment
to reflect current usage (stranded-comment cleanup, not post-review)
- modes.ts: merge duplicate report_progress steps (8 + 10) into single
step 9, fix "optimizatfixons" typo
- wiki/post-cleanup.md: document checklist detection regex
Made-with: Cursor
* collapsible completed todos in final progress, hide set_output outside standalone mode
- add renderCollapsible() to TodoTracker, append completed task list as
<details> section when agent calls report_progress
- cancel tracker after agent's final report_progress so it doesn't
overwrite with raw checklist
- conditionally register SetOutputTool only in standalone mode or when
output_schema is provided
- remove unconditional set_output instruction from orchestrator task section
- update Summarize/SummaryUpdate mode guidance to not reference set_output
Made-with: Cursor
* show completion count in collapsible task list summary
Made-with: Cursor
* only count completed (not cancelled) in collapsible task list summary
Made-with: Cursor
* reinforce concise summary prompting across system prompt, modes, and tool description
Made-with: Cursor
* address review feedback: wasUpdated bypass, tracker false-positive, race condition
- remove wasUpdated=true from cleanup paths so handleAgentResult correctly
detects genuinely silent runs
- add hadProgressComment to ToolState as immutable snapshot for the safety check
- use todoTracker.hasPublished instead of enabled for stranded-comment cleanup
- serialize onUpdate calls via inflightPromise chain with post-cancel guard
- add settled() to wait for in-flight updates before writing final summary
Made-with: Cursor
* address round-2 review: hasPublished after success, finalSummaryWritten flag
- set hasPublished only after onUpdate resolves (not before) so failed
writes are not counted as published
- add finalSummaryWritten flag to ToolState, set after successful
non-plan reportProgress; decouple cleanup detection from
todoTracker.enabled so it survives API failures where cancel() ran
but the write didn't succeed
Made-with: Cursor
* add repo learnings feature with edit history
introduces a new "Learnings" section in the repo console where agents can
persist operational knowledge (setup steps, test commands, conventions) at
the end of runs via an MCP tool. users can also edit learnings manually.
- add `learnings` field to Repo model and `LearningsRevision` audit table
- add `update_learnings` MCP tool for agents to persist repo knowledge
- integrate learnings into prompt assembly as REPO LEARNINGS section
- add learnings step to mode guidance (Build, AddressReviews, Plan, Fix, Task)
- add PATCH /api/repo/[owner]/[repo]/learnings endpoint (JWT auth)
- add GET /api/repo/[owner]/[repo]/learnings/history endpoint (Clerk auth)
- add LearningsSection component with textarea, save-on-blur, and history modal
- record revision history with actor tracking (agent vs user) and pruning (50 max)
- gate UI behind owner === "pullfrog" for internal dogfooding
Made-with: Cursor
* fix prisma enum import path for LearningsActor
Made-with: Cursor
* simplify learnings schema: remove LearningsActor enum, store model name directly
the actor/actorName split was unnecessary — learnings are only written by
agents so the revision table just needs a model column. removes all user
editing concepts from schema, API, and frontend.
Made-with: Cursor
* fix migration: add separate migration instead of rewriting existing one
restores original learnings_revisions migration and adds a new migration
that drops actor/actorName columns, backfills model from actorName, and
drops the LearningsActor enum.
Made-with: Cursor
* polish learnings feature: rename to Repo Intelligence, fix atomicity, fix review skip
- rename user-facing "learnings" to "Repo Intelligence" (UI, prompt section, wiki, sidebar)
- simplify description to "Automatically discovered by the agent across runs."
- wrap repo.update + revision create in $transaction for atomicity
- refactor recordLearningsRevision to pruneLearningsRevisions (prune-only)
- fix empty review skip: don't block APPROVE reviews with no body
- fix broken docs anchor: #free-options → #free-models
- update agent guidance to require flat bullet list format with pruning
- add accessibility: aria-expanded, sr-only loading, output element
- add chevron rotation, stale data clear on modal close, max-h scroll
- trim + length-limit model field, remove type cast, restore pre-existing comment
- update wiki prompt examples with actual bullet-formatted content
- update model test snapshot
Made-with: Cursor
* fix stale free model name in docs, rename utility file to match export
- docs/keys.mdx: MiMo V2 Flash → MiMo V2 Pro (matches model code change)
- rename recordLearningsRevision.ts → pruneLearningsRevisions.ts
Made-with: Cursor
* Add skill, .neon
* polish learnings UI and remove verbose log
- learnings code block: read-only appearance with muted text, copy button, rounded corners
- history modal: full-width rows with cursor-pointer, chevron moved to right, no preview text
- drop noisy update_learnings log line
Made-with: Cursor
* inject learningsStep into all modes, drop seed script, soften revision styling
Made-with: Cursor
* Drop seed
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* add mode instructions and restructure dashboard sidebar
- add modeInstructions JSONB field to Repo model for per-mode user instructions
- thread modeInstructions through settings API, run-context API, RepoSettings, ToolContext, and selectMode runtime
- merge user-defined mode instructions with hardcoded orchestrator guidance, with IncrementalReview inheriting from Review
- reduce visible built-in modes from 7 to 4 (Build, Review, Plan, Fix) with editable Instructions textareas
- add TRIGGERS group header to sidebar above Mentions, Pull requests, Issues
- add wiki/modes.md documenting triggers and modes conceptual model
Made-with: Cursor
* fix leaping comment deletion and address review feedback
- wrap post-createReview operations in try/finally so deleteProgressComment
runs even when updateReview or reportReviewNodeId throws
- add parseModeInstructions runtime guard to filter non-string values
from the JSONB field before passing to buildOrchestratorGuidance
- add useEffect sync for localInstructions when props change
- guard onBlur to skip save when instructions haven't changed
- update wiki/modes.md to reflect V2 is implemented (no longer "proposed")
Made-with: Cursor
* harden review cleanup, fix type cast, stabilize mode instructions state
- wrap deleteProgressComment in try/catch inside finally to prevent masking original errors
- replace `as Record<string,string>` cast with runtime parseModeInstructions + useMemo
- fix wiki dual-prompt table to reflect mode.prompt fallback status
Made-with: Cursor
* fix wiki tense and heading ambiguity from PR review
Made-with: Cursor
* fix review "edited" badge by using pending review + submit flow
create review as PENDING first (no event/body), build the footer with
the now-known review ID, then submitReview with the full body. single
atomic publish — no updateReview edit needed.
Made-with: Cursor
* add post-agent follow-up re-review dispatch
After the agent exits, check if PR HEAD moved past the reviewed commit
and dispatch a follow-up re-review. This closes the gap where push
webhooks are suppressed during in-flight reviews.
Made-with: Cursor
* add silent flag to follow-up re-review dispatch
Made-with: Cursor
* restructure dashboard for consistency and clarity
- consolidate tools into single grouped card (was 4 separate cards)
- merge coding + autofix CI into one section
- remove redundant trigger section descriptions
- add bidirectional crosslinks between modes and triggers
- inline instruction links (review/plan/build) into descriptions
- add save status indicators to all sections
- restructure flags with grouped built-in/custom cards
- flatten sidebar (remove dividers and group headers)
- tighten all descriptions
Made-with: Cursor
* update PR screenshots for new dashboard layout
Made-with: Cursor
* extend review context inline instead of dispatching new workflow
when commits are pushed during a review, the agent now handles them
inline: create_pull_request_review detects HEAD movement, returns
instructions to pull and review the incremental diff, and the agent
submits a second review covering only the new changes. this avoids
the cost of spinning up a full new workflow run.
also fixes a bug where reviewedSha was set to the submission HEAD
(current) rather than the checkout HEAD (what was actually reviewed),
which caused commits pushed between checkout and submission to be
silently missed by postReviewCleanup.
the workflow dispatch is kept as a safety net for agent timeout/error.
Made-with: Cursor
* polish dashboard UI: fix debug markers, crosslinks, title consistency, descriptions
- remove all red debug borders/labels and CM component
- remove all inline style={{}} debug outlines from crosslinks
- fix ambiguous crosslinks: Build→"Coding ↓", Plan→"Enrich issues ↓"
- add missing "Edit build instructions ↑" backlink on Auto-address reviews
- normalize card title weight to text-sm font-semibold across all cards
- rename "Default" subcard to "Setup" with broader description
- fix Mentions description to imperative tone
- broaden Flags section description to cover built-in and custom
- remove useless fragments in ModesSection and ToolsSettings
- restructure Agent section: remove ConsoleSection wrappers, add sidebar indent support
Made-with: Cursor
* extract PR quick links as standalone card, consistent with issues
- PR quick links is now its own card under Reviews (was a sub-toggle inside Review PRs disabled state)
- Review PRs OFF sets prCreated="none" instead of auto-falling back to "links"
- Review PRs card hides sub-toggles when disabled (re-review/approve don't apply)
- Both PRs and Issues now have identical Quick links card structure
Made-with: Cursor
* update reviews screenshot with standalone quick links card
Made-with: Cursor
* polish dashboard UI: revert quick links to inline toggles, fix fonts and spacing
- revert standalone PR/issue Quick Links cards back to inline toggles inside
Review PRs and Enrich Issues cards (fixes prCreated state coupling bug)
- restore original font-medium card titles across all trigger/settings cards
- fix sidebar: add CONSOLE heading, remove nested indentation, remove truncation
- right-justify Enrich Issues mode dropdown, group description with label
- move instructions links inline with behavior descriptions
- replace text save indicators with icon spinner/checkmark
- standardize section title spacing, move footer below danger zone
Made-with: Cursor
* fix formatting for biome lint
Made-with: Cursor
* address PR review feedback: cleanup guard, shared util, wiki update
- clear ctx.toolState.review after read to prevent double-execution of postReviewCleanup
- forward authorPermission in safety-net re-review dispatch
- extract parseModeInstructions to utils/schemas/modeInstructions.ts
- update wiki/modes.md: remove stale v1/v2 language, fix dashboard layout
- add typecheck to pre-push hook
Made-with: Cursor
* add action typecheck to pre-push, fix exactOptionalPropertyTypes errors
Made-with: Cursor
* fix duplicate actuallyReviewedSha from rebase
Made-with: Cursor
* remove PR screenshots
Made-with: Cursor
* add label/textarea association for mode instruction accessibility
Made-with: Cursor
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Prevents the agent from calling select_mode multiple times, which caused
it to chain modes (e.g. Plan then Build) when the user only asked for a
plan. Also removes the Plan orchestrator guidance that explicitly
encouraged switching to Build after planning.
Closes#394
Made-with: Cursor
* add incremental re-review on new PR commits
When new commits are pushed to a PR that Pullfrog has previously reviewed,
automatically perform a focused re-review on only the new changes. Includes
a supersede mechanism to abort stale in-flight reviews on rapid pushes, a
new IncrementalReview mode with incremental diff + prior-feedback awareness,
and a PRReview tracking model so re-review fires for both auto-reviewed and
manually-triggered PRs. Reviews now always submit (APPROVE when clean).
Co-authored-by: Cursor <cursoragent@cursor.com>
* simplify re-review eligibility and add summary to incremental reviews
Remove the path1/path2 distinction for re-review eligibility — now simply
requires prReReview=enabled and a prior Pullfrog review on the PR. Show
the re-review toggle regardless of prCreated setting. Add a top-level
summary body to incremental reviews for consistency with full reviews.
Co-authored-by: Cursor <cursoragent@cursor.com>
* replace superseded polling with server-side in-flight dedup and add prApproveEnabled setting
Made-with: Cursor
* add armstrong cursor command
Made-with: Cursor
* update armstrong
* feat: Pullfrogger game v1 (#378)
* Frogger game basis code (CC0 1.0 Unversal license).
* Initial React port.
* fix props for Sprite.
* fixed context issue in FroggerGame.
* Fixed format.
* Fixed lint issue in FroggerGame.
* Restoring LeapingLoader, using FroggerGame as a new fallback in Suspense.
* feat: Display toast when URL ready.
* Add props constraints on Sprite.
* feat: frog sprite.
* fix: zoom and alignment.
* fix: extract const.
* fix: mv types.
* fix: mv game into index.tsx file.
* fix: replacing deprecated event prop which with key.
* feat: Log sprite.
* feat: turtle sprite.
* Adjusting game colors.
* feat: sprites for the cars.
* rm primitive sprites.
* fix: bulldozer sprite position.
* Adjusting colors.
* Shape constraints.
* rm original.
* minor: naming, cleanup.
* fix: renderers dict.
* fix: adjusting and renaming racer.
* fix renderer binding for scored frogs.
* feat: responsive layout with gap below and maintained aspect ratio.
* grammar fix.
* feat: road lane dividers.
* feat: using AbortController to cleanup events.
* fix: cleanup and shortening.
* feat: extracting drawGameBackground.
* feat: initObstacleRows and updateAndDrawObstacles helpers.
* feat: initFroggers helper.
* feat: drawFroggers helper.
* feat: checkForCollision helper.
* feat: makeKeydownHandler helper.
* feat: listenToKeyboardEvents helper.
* mv cleanup into drawGameBackground.
* fix: cleanup.
* feat: better adjustBrightness helper.
* Polling on the page.tsx side, restoring timeout and fallbacks, dynamic link msg.
* FEAT: wait for workflow to complete and notify additionally with a big link (incl.db migration).
* Fix: larger title, shorter link.
* fix(hook): Writing completedAt from hook.workflow_run.updated_at according to suggestion.
* fix(loader): rm unused props from WorkflowRunClientProps as suggested.
* fix(loader): mv id=dev case into the page.
* fix(DNRY): mv PollStartedResult and PollCompletedResult types.
* fix(DNRY): reusing drawEllipse() helper in Sprite.
* fix(style): reordering methods by priority in Sprite.
* fix(frogger): rm empty rows from canvas, square game.
* feat: game canvas rounded corners.
* fix(DNRY): extracting SpriteShape type for faster reference.
* fix: rm target from links, use same window.
* add incremental re-review on new PR commits
When new commits are pushed to a PR that Pullfrog has previously reviewed,
automatically perform a focused re-review on only the new changes. Includes
a supersede mechanism to abort stale in-flight reviews on rapid pushes, a
new IncrementalReview mode with incremental diff + prior-feedback awareness,
and a PRReview tracking model so re-review fires for both auto-reviewed and
manually-triggered PRs. Reviews now always submit (APPROVE when clean).
Co-authored-by: Cursor <cursoragent@cursor.com>
* replace superseded polling with server-side in-flight dedup and add prApproveEnabled setting
Made-with: Cursor
* consolidate workflow_run completed handling and track completedAt
Removes the duplicate exported handleWorkflowRunCompleted in favor of
the private one, merges status + orphan resolution logic into a single
path, and sets completedAt on both normal completion and orphan cancel.
Made-with: Cursor
* fix rebase conflict resolution: restore eligibility logic, incremental review summaries, and exhaustiveness check
- replace deleted hasPullfrogReviewedPR call with WorkflowRun.findFirst
(the utility file was removed by the dedup improvements commit)
- restore IncrementalReview summary body in modes.ts and selectMode.ts
(lost during ca0168b conflict resolution; origin had re-added them via f98f902)
- use switch + satisfies never for workflow_run event dispatch
- lowercase comments per project conventions
Made-with: Cursor
* fix workflow-run polling architecture and improve incremental review prompts
move polling loops from server actions to client to avoid serverless timeouts
(pollForCompleted ran up to 600s in a single invocation). each server action
is now a single DB check; client drives retries. also fix misleading prompt
text about incremental diff scope and remove dead code in handleWebhook.
Made-with: Cursor
* await reportReviewNodeId to eliminate race condition and webhook sleep
- refactor reportReviewNodeId from fire-and-forget to async/awaited,
guaranteeing the dedup signal lands before the tool returns
- remove the 5-second grace period sleep in the synchronize webhook
handler (no longer needed with the awaited PATCH)
- update IncrementalReview guidance to use get_review_comments for
detailed prior line-level feedback instead of just review summaries
- remove dead fallbackUrl field from CheckStartedResult type
Made-with: Cursor
* fix rebase artifacts: broken triggeringIssue reference, review formatting, and prompt wording
Made-with: Cursor
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Anna Bocharova <robin_tail@me.com>
* pin all CLI installations to explicit versions and use pro models by default
- codex: pin @openai/codex to 0.101.0 (was "latest")
- opencode: pin opencode-ai to 1.1.56 (was "latest")
- gemini: pin gemini-cli to v0.28.2 via new tag param on installFromGithub
- cursor: pin to 2026.01.28-fd13201 via direct tarball download (replaces curl install script)
- add installFromDirectTarball to install.ts for versioned tarball URLs
- gemini auto effort now uses pro-preview instead of flash-preview
- test runner model overrides updated to use pro-preview consistently
Co-authored-by: Cursor <cursoragent@cursor.com>
* increase activity timeout
* fix delegation timeout
* fix delegate timeouts
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* pin all CLI installations to explicit versions and use pro models by default
- codex: pin @openai/codex to 0.101.0 (was "latest")
- opencode: pin opencode-ai to 1.1.56 (was "latest")
- gemini: pin gemini-cli to v0.28.2 via new tag param on installFromGithub
- cursor: pin to 2026.01.28-fd13201 via direct tarball download (replaces curl install script)
- add installFromDirectTarball to install.ts for versioned tarball URLs
- gemini auto effort now uses pro-preview instead of flash-preview
- test runner model overrides updated to use pro-preview consistently
Co-authored-by: Cursor <cursoragent@cursor.com>
* increase activity timeout
* fix delegation timeout
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
