Compare commits

...

18 Commits

Author SHA1 Message Date
Colin McDonnell 17b610e1a1 bump action version to 0.1.1 2026-05-08 21:32:06 +00:00
Colin McDonnell ca913c76ea spawn: kill process group + heartbeat subagent activity (#631)
* spawn: kill process group + heartbeat subagent activity

two compounding bugs produced zombie agent runs that stalled until the
GitHub-Actions job-level timeout (observed on PR #622, run 25577068620).

1. SIGKILL hit the wrong process. node_modules/opencode-ai/bin/opencode
   is a Node shim that spawnSyncs the native opencode-<plat>-<arch>
   binary with stdio:"inherit". our spawn() ran without detached, so
   child.kill("SIGKILL") killed only the shim. the native binary was
   reparented to PID 1, kept holding our stdout pipe via inherited fds,
   and child.on("close") never fired — leaving the agent promise
   pending past the 5min outer safety-net timer ("agent still pending
   5min after inner activity kill — forcing exit") and the grandchild
   running until the runner timed out.

   fix: SpawnOptions gains killGroup; when set, we spawn detached and
   route all kill paths (timeout, activity timeout, ctrl-c) through
   process.kill(-pid, signal). opencode + claude opt in.

2. inner activity timer false-fired during long task subagents.
   opencode's `task` tool encapsulates subagent execution in-process —
   subagent-internal events don't reach the parent NDJSON stream — so
   the parent looked idle for the full subagent duration even when
   real work was happening, and the 5min DEFAULT_ACTIVITY_TIMEOUT_MS
   would fire mid-subagent.

   fix: SpawnOptions gains externalActivitySource; the timer fires on
   min(local stdout idle, external idle). opencode passes getIdleMs()
   from the global activity tracker and runs a 30s heartbeat
   (markActivity()) while at least one task dispatch is in flight.

action/utils/subprocess.test.ts covers both: a bash+sleep grandchild
that proves close fires <10s with killGroup, and externalActivitySource
keeping the timer armed during 8s of stdout silence.

* opencode: suspend activity timer instead of heartbeat during subagent runs

addresses review on prior commit: replace the 30s markActivity()
heartbeat with a boolean isPausedExternally predicate keyed off
opencode's existing taskDispatchByCallID + pendingTaskDispatches.
no fake activity, no race window between a 30s tick and a subagent
that finishes between ticks.

while the predicate returns true, spawn's activity check skips the
kill decision *and* advances lastActivityTime so a clean unpause
can't fire on a stale baseline. tests cover both the suspended case
(8s of stdout silence + activityTimeout=1s but paused → process
exits cleanly) and the resume case (paused for 500ms then unpaused
→ 30s sleep gets killed by activity timeout as normal).
2026-05-08 21:29:22 +00:00
Colin McDonnell 20d4b12522 bump action version to 0.1.0
document direct-to-main exceptions in AGENTS.md (version bumps and
other release-trigger commits when the user explicitly says "push to
main").
2026-05-08 21:26:53 +00:00
Colin McDonnell ec43c0e0d1 router: fix bugs from PR #616 review (#625)
Three real defects flagged in the post-merge review of #616, plus one cheap
hardening:

1. OpenCode `limit.output` override was a silent no-op on opencode-ai@1.1.56.
   Top-level `limit.output` has no read site in OpenCode (verified against
   the v1.1.56 source: `OUTPUT_TOKEN_MAX = Flag.OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX
   || 32_000` in session/llm.ts; per-model `model.limit.output` has its own
   scope). Plumbed via `OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX=5000` env var
   on the OpenCode spawn instead. Drops dead `OpenCodeConfig.limit?` type
   field and the corresponding config write in `buildSecurityConfig`. This
   was the headline mechanism of #616 — without the env var, the upfront
   `max_tokens` reservation stayed at 32_000 and low-wallet runs continued
   failing the way #616 was supposed to prevent.

2. Phantom auto-reload buffer for detached-card accounts. DELETE
   /payment-method clears `stripeCustomerId` but leaves `autoReloadEnabled`
   intact, so an account with welcome-credit residue and a detached card
   could mint a key with `keyLimitCents = balance + autoReloadAmountCents`
   ($50 default, schema-cap $100K) of free spend headroom we have no way
   to bill. Conjunctive `account.autoReloadEnabled && hasCard` in the
   buffer selection closes this. Defense-in-depth follow-up worth doing:
   clear `autoReloadEnabled` in the card-detach handler.

3. The autoReloadEnabled 402 branch fired for phase-1 noop paths
   (`!stripeCustomerId`, `reloadAmountCents < 50`, `balance >= threshold`)
   where `result.failure == null`, returning `"insufficient balance"` with
   no actionable code. Gated on `result.status === "failed"` so non-charge
   paths fall through to the `hasCard` / no-card branches and emit
   `router_balance_exhausted` / `router_requires_card` instead.

4. (cheap) `ROUTER_KEYLIMIT_EXHAUSTED_PATTERN` now uses `/is` instead of
   `/i` so `.*?` crosses newlines. Defends the BillingError reclassification
   against any upstream layer that wraps the OpenRouter error onto multiple
   lines. Trivial.

Test plan: 488/488 unit tests pass (1 new test for newline regex behavior).
2026-05-08 21:02:38 +00:00
Colin McDonnell 93cc7b1a44 show effective model in agent comment/review footers (#618)
`toolState.model` was set only to `payload.model` (the stored slug, often
undefined for router/oss runs that derive the target from `proxyModel`).
the footer's "Using `…`" segment is gated on a truthy model, so router
runs on repos without an explicit model setting shipped reviews/comments
with no model badge — e.g. PR #614's review showed no model despite
running `openrouter/anthropic/claude-opus-4.7` via proxy.

now mirror the priority used by `resolveModelForLog` and `isGeminiRouted`:
`payload.proxyModel ?? resolvedModel ?? payload.model`. also reverse-look
up by `resolve`/`openRouterResolve` in `formatModelLabel` so a proxy
target like "openrouter/anthropic/claude-opus-4.7" still renders as
"Claude Opus".
2026-05-08 20:59:09 +00:00
pullfrog[bot] 851e49e2d7 action: retry transient GitHub 422 "internal error" on review submission (#610)
* action: retry transient GitHub 422 "internal error" on review submission

GitHub sometimes 422s POST /pulls/{n}/reviews with body
"An internal error occurred, please try again." — a server-side hiccup
that the existing 422 handler framed with the generic
"likely causes (1)(2)(3)" prompt listing affected comments. the agent
dutifully refetched the diff, dropped comments, and resubmitted, hitting
the same transient error on a shifting affected-comments list until
GitHub accepted. some runs logged 8+ spurious retries with ~11 minutes
of wall-clock, dropping valid inline comments along the way.

detect the transient body explicitly, retry in-tool twice with 1s/3s
backoff, and surface a distinct error on exhaustion that tells the agent
this is a GitHub-side issue — do not modify inline comments, wait and
retry or fall back to a body-only review. closes #584.

* action: use retry util for transient review 422, drop isTransientReviewError tests

---------

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
2026-05-08 20:33:01 +00:00
Colin McDonnell 4101df566b router: decouple per-run key budget from wallet, add overdraft buffer (#616)
Replaces today's `keyLimitUsd = min(walletBalance, $25)` with population-aware
buffers so users can use 100% of their credits before being paywalled, and
opaque mid-run "more credits" failures (e.g. https://github.com/pullfrog/app/actions/runs/25531633203)
get a clear PR comment instead of a generic stack-trace dump.

Policy matrix:
- Auto-reload accounts: `wallet + autoReloadAmountCents` (default $50, no cap)
- Card + no-autoreload: `wallet + $5` overdraft buffer
- No card: `wallet` (no buffer; existing zero-balance 402 stays)
- OSS: `$10` (unchanged)

Removes the $25 per-run cap entirely. Long Build runs at high-balance
accounts no longer silently cap at $25.

Other changes:
- Classify mid-run OpenRouter "requires more credits, or fewer max_tokens"
  errors as `router_keylimit_exhausted` BillingError so users get an
  actionable PR comment.
- Override OpenCode `max_tokens: 32000` default to `5000` via
  OpenCodeConfig.limit.output. Drops Opus per-call upfront budget reservation
  from ~$2.40 to ~$0.38 — what makes low-wallet runs viable at all.
- Switch `findInitialComment` and `findExistingPaywallComment` to GraphQL
  `issueOrPullRequest(number:) { comments(last: 100) }` (single round trip,
  actually returns newest-100; REST listComments doesn't support sort/direction).
  Also fixes a latent `comments.find()` returning the OLDEST match instead
  of the most recent — now selects max(databaseId).
- Wrap `syncAccountUsage` in `prisma.$transaction` with `SELECT ... FOR UPDATE`
  on the account row. Pre/post-balance reads inside the transaction enable
  deterministic low-balance edge detection (currently logs; will push the
  outreach.low_balance task once #592 lands).

Plan: .cursor/plans/router-low-balance-paywall.plan.md (in companion wiki-billing branch)
2026-05-08 20:15:47 +00:00
Colin McDonnell 9d04cad360 drop legacy summaryCommentNodeId column (#617)
Was retained on `workflow_runs` after PR #568 replaced the comment-based
summary path with the snapshot architecture, with a "kept for backfill of
pre-snapshot runs" annotation. No backfill is planned: pre-snapshot summary
comments were written in the user-facing PR_SUMMARY_FORMAT (TL;DR + key
changes blockquote + before/after sections), not the agent-context
functional-summary format the snapshot now expects. Backfilling them would
prime new runs with the wrong shape and pollute the agent context. Old
comments stay on github.com as historical artifacts; the column on the DB
row is dead weight.

Strips the field from:
- prisma schema + new migration `20260508190000_drop_summary_comment_node_id`
- `app/api/workflow-run/[runId]/route.ts` STRING_FIELDS allowlist
- `action/utils/patchWorkflowRunFields.ts` type union + STRING_KEYS
- `utils/db/selectActiveWorkflowRuns.ts` select clause
- `utils/github/enrichWorkflowRunsWithArtifactUrls.ts` node-id type, URL
  resolution, collectUniqueNodeIds + urlsForRun
- `utils/webhooks/handleWorkflowRunWebhook.ts` two select clauses, the
  hasRecordedArtifact param, and the orphaned-leaping-comment alert text
- `components/RunArtifactPills.tsx` ArtifactKey union + ARTIFACT_KEYS +
  switch cases (drops the "View summary" chip from the workflow run list)

Verified: pnpm typecheck clean, pnpm lint clean (537 files), action build
clean. Dev DB reset against production parent and the migration applied
cleanly — column is gone from the workflow_runs table.
2026-05-08 19:47:38 +00:00
Colin McDonnell e4e93ea6d3 PR summary as agent-edited tmpfile snapshot (#568)
* PR summary as agent-edited tmpfile snapshot

Replaces the comment-based PR summary path (and the in-progress
update_pr_summary tool from #534) with a snapshot file the agent edits in
place during Review / IncrementalReview / pr-summary Task runs.

The server seeds the tmpfile with the previous snapshot (incremental) or a
stable scaffold (first run), exposes the path via select_mode, and reads it
back at end-of-run to persist to WorkflowRun.summarySnapshot and (when the
prSummaryComment toggle is on) splice into the PR description body.

Why a tmpfile rather than a tool call: incremental snapshot edits are
output-token-cheap when the agent uses native file-editing tools, and
range-diff cleanly across runs because section headings are stable. The
agent never has to regurgitate the full snapshot to update it.

Gating: snapshot generation is opt-in via either prSummaryComment="enabled"
(splice into PR body) or prReReview="enabled" (snapshot feeds future
incremental review runs as context). Users who disable both pay nothing
end-to-end — no seeding, DB write, or body splice.

Behavior changes:
- Drop the Summarize mode and the Summary comment type entirely; the
  rolling summary is no longer a separate run shape.
- pull_request_synchronize with re-review off and summary on still
  dispatches a silent pr-summary Task, but it edits the snapshot file
  instead of posting a fresh comment.
- /api/repo/.../pr/.../summary-comment now returns
  { snapshot: string | null } from the DB instead of fetching a comment via
  GraphQL. URL kept stable so deployed older actions degrade gracefully.
- summaryCommentNodeId is retained on WorkflowRun for legacy data and a
  future backfill of pre-snapshot comment-based summaries.

Supersedes #534. The commit-tool/sub-agent direction in that PR is
abandoned in favor of this file-based shape.

* address review pass #1: synchronize fallback, splice idempotency, docs

* address review pass #2: in-flight skip should not race summary fallback

* address review pass #3: signal-handler flush, doc clarifications

* address review pass #4: in-flight persist promise + bounded body-splice timeout

* address review pass #5: defensive catch on persist worker, doc nit

* add summary-stale post-run gate

When generateSummary is set, we capture the bytes of the seeded snapshot
file and pass them to the agent's post-run loop alongside the file path.
After each agent attempt, the loop diffs the current file against the
seed; if they're byte-identical the agent never touched it, and we nudge
once via a resume turn (similar to the dirty-tree gate, but soft and
fire-once so smaller models that legitimately decide no edit is warranted
don't burn the retry budget).

Mostly defends against forgetful smaller models on the Review path —
their mode prompt asks them to edit the snapshot file, but the
multi-step instruction can fall through when the diff is large.

* trigger: retry vercel preview build

* fix(action): drop unused re-export that pulled node:fs/promises into next bundle

action/internal/index.ts was re-exporting DEFAULT_PR_SUMMARY_INSTRUCTIONS
from action/utils/prSummary.ts, but nothing in the next.js app imports
it. prSummary.ts uses node:fs/promises, and pullfrog/internal is aliased
into the next bundle by next.config.ts, which made turbopack try to
resolve node:fs/promises in client chunks and fail with:

  the chunking context (unknown) does not support external modules
  (request: node:fs/promises)

drop the re-export — selectMode.ts (the only real consumer) already
imports it directly from action/utils/prSummary.ts.

* firewall PR summary snapshot from user instructions; resurrect rich format for Review

The agent-internal snapshot (the markdown file the agent edits in place across
runs) is exclusively durable context for future agent runs — user-supplied
summarization instructions warp it and degrade that context. Drop the
prSummaryCommentInstructions read path end-to-end:

- handleWebhook: stop reading prSummaryCommentInstructions, stop passing
  prSummaryInstructions through dispatch options
- action payload + ToolState + selectMode addendum: drop the instructions
  appendix; the snapshot prompt is fixed, not user-shaped
- TriggersSettings: drop the InstructionsEditor for prSummaryCommentInstructions
- prSummary.ts: reframe DEFAULT_PR_SUMMARY_INSTRUCTIONS as agent-targeted
  (durable context, not human-facing prose)

Prisma columns (prSummaryComment, prSummaryCommentInstructions) and the
matching zod schema entry stay for graceful retreat.

Separately, resurrect PR_SUMMARY_FORMAT (deleted along with the Summarize mode
in the original PR) and wire it into Review mode only. Initial PR reviews now
include a structured summary section in the review body using the rich format
(TL;DR, key changes, ## sections with before/after, file-link trails).
IncrementalReview keeps its existing terser bullet-list shape since re-review
bodies are deltas, not introductions. The user-facing review summary and the
agent-internal snapshot are deliberately separate artifacts with separate
prompts and zero shared content.

* address review comments: prompt self-consistency + stale-doc cleanup

PR 568 self-review (4232488109) flagged a self-contradiction the firewall
commit introduced and three stale doc references that survived.

- action/modes.ts: Review-mode step 2's trivial-PR shortcut said `submit
  "Reviewed — no issues found." per step 5`, but step 5's rewrite removed
  exactly that preamble. Aligned both: trivial PRs and no-actionable-issues
  PRs now produce a body that opens with "No new issues found." followed by
  the PR summary, so the user gets the headline up front and still sees what
  was reviewed.
- docs/pr-reviews.mdx: dropped the "customize the summary style with Summary
  instructions in the console" sentence (the editor was removed in the
  firewall commit). Replaced with a note that the snapshot uses Pullfrog's
  built-in format and is not user-customizable.
- wiki/prompt.md, wiki/modes.md: rewrote the snapshot-prompt entries to
  reflect the firewall — DEFAULT_PR_SUMMARY_INSTRUCTIONS is the entire
  prompt, prSummaryCommentInstructions is no longer wired in.

* drop orphaned prSummaryCommentInstructions column

Prod audit (455 repos): 5 non-null rows on a single account, all containing the
literal placeholder text from the InstructionsEditor we removed in the firewall
commit. No account has an intentional preference set, so silent-ignore (the
keep-for-retreat option) costs us nothing meaningful while leaving an orphan
column in the schema. Drop it.

- prisma/schema.prisma: remove the column
- prisma/migrations/20260506000000_drop_pr_summary_comment_instructions:
  ALTER TABLE ... DROP COLUMN
- utils/schemas/triggers.ts: drop the matching zod entry

* drop body splicing; snapshot is internal-only

User-visible PR summarization continues to ship in Review and IncrementalReview
review bodies (which already render PR_SUMMARY_FORMAT and "Reviewed changes"
respectively). The snapshot tmpfile is now purely durable cross-run agent
context — seed, edit-in-place, save to DB, feed the next run. Massive
simplification: the body splice mechanics, the two-toggle gating matrix, the
summaryHandlingCovered race tracking, and the synchronize summary-only Task
fallback all go away.

Code:
- prSummary.ts: drop splice/strip/marker code (`splicePrSummary`,
  `stripExistingSummaryBlock`, `buildSummaryBlock`, `extractPrSummary`,
  PULLFROG_SUMMARY_START/END). keep scaffold, instructions, seed/read.
- main.ts: rename persistAndPostSummary -> persistSummary; collapse to a
  single DB PATCH. drop pulls.get/pulls.update, drop AbortSignal timeout,
  drop in-flight promise machinery, drop prSummaryToBody plumbing.
- ToolState: add summarySeed (replaces local var in main.ts so persist can
  compare). drop prSummaryToBody and summaryPersistInFlight.
- persistSummary now compares against the seed and skips the DB write
  with a warning when unchanged — saving the seed verbatim is either a
  no-op or persists the placeholder scaffold, neither useful.
- postRun.ts: when summary-stale is the only failing gate and the resume
  turn itself fails, restore the pre-resume successful result and break.
  symmetric with the existing reflection-failure preservation. summary-stale
  can no longer flip a successful run to failed.

Webhook:
- pull_request_opened: generateSummary follows prReReview only (the snapshot
  has no consumer when re-review is off).
- pull_request_synchronize: collapses to "if prReReview enabled, dispatch
  IncrementalReview". the summaryHandlingCovered flag, the same-SHA/in-flight
  coordination it was protecting, and the summary-only Task fallback all
  delete cleanly.

UI / config:
- drop SummarizePRsTrigger (the toggle gated body splice; with that gone
  it has no behavior). drop sidebar entry, console import, Text icon import.
- drop prSummaryComment from triggers zod schema, prisma schema, preview
  settings script.

Migration: squash the two existing migrations into one timestamped
20260507000000_pr_summary_snapshot covering all three column changes
(add summarySnapshot on workflow_runs, drop prSummaryCommentInstructions
and prSummaryComment on repos). repo convention is one migration per PR.

Action: bump 0.0.203 -> 0.0.205 (payload contract changed: prSummaryToBody
removed; main is at 0.0.204).

Out-of-diff cleanup:
- review.ts:190 + review.test.ts:651 — "Reviewed — no issues found." ->
  "No new issues found." to match the canonical body in modes.ts.

Verified: pnpm typecheck clean, pnpm lint clean, postRun + review tests
pass, dev DB reset against production and the squashed migration applied
cleanly (summarySnapshot present, prSummaryComment / prSummaryCommentInstructions
both gone).

* re-orient snapshot toward functional summary; drop prior-review-feedback section

Empirical audit on preview-568 PR #5 showed the snapshot IS load-bearing
for the orchestrator: lens-dispatch prompts on incremental runs carried
forward context from the snapshot's risk register (e.g. "the JSDoc
explicitly scopes to code points — do not flag grapheme-cluster issues"
on the surrogate-pair fix run, "consistency with native padStart" on the
padStart-added run). The orchestrator was reading the snapshot, reasoning
about it, and using it to anti-prime / focus subagents — exactly the
high-leverage path. My earlier "snapshot is write-only" claim was wrong.

The shape, however, was steering it toward review-history-log instead of
functional summary. This commit re-orients:

- prSummary.ts: replace the four-section scaffold (~580 chars of placeholder
  italics under "What this PR does / Key changes / Risk / Reviewed in prior
  runs") with a minimal seed (~150 chars: just a header + a one-line
  comment about what the file is for). different PRs warrant different
  organization; forcing a refactor and a feature into the same template
  is procrustean. minimal seed also makes the unchanged-from-seed gate
  in persistSummary more sensitive.

- selectMode.ts addendum: rewrite around three principles. (1) the snapshot
  is a FUNCTIONAL summary of what the PR does and the risks it carries,
  not a chronological review log — commit history can already be
  reconstructed from list_pull_request_reviews. (2) the orchestrator should
  USE the snapshot during triage and dispatch — concrete example given of
  carrying snapshot context into subagent lens prompts. (3) structure is
  the agent's call; stable headings make snapshots range-diff cleanly when
  they fit, but riff when they don't.

- modes.ts IncrementalReview: drop the "Prior review feedback" checklist
  from the user-facing review body (step 6b gone, step 7 ELSE IFs cleaned
  up). It duplicated content that's already covered by the Reviewed-changes
  bullets and tracked durably in the snapshot for the next agent run; in
  the user-facing body it was noise. step 3 still fetches prior reviews
  but its role is now just filtering aggregation in step 5, not rendering.

- AGENTS.md: codify "no follow-ups" rule. when an issue is identified
  during code review, fix it in this PR — PR scope does not constrain
  quality. follow-up TODOs are forbidden as a substitute for doing the
  work now.

Empirical evidence supporting the re-orientation:

- Run 25568912293 (PR#5 incr1, surrogate-pair fix): orchestrator's
  correctness lens dispatch said "Do NOT flag grapheme-cluster issues
  — the JSDoc scopes to code points." The grapheme-cluster framing was
  not in the diff; it was downstream of the snapshot's prior risk-section
  framing of truncate's contract. Snapshot influencing dispatch.

- Run 25569054779 (PR#5 incr2, padStart added): orchestrator's correctness
  lens dispatch enumerated edge cases including "consistency with native
  String.prototype.padStart contract" and "fill = multi-code-point string
  (e.g. emoji)". Both threads carried over from the snapshot's prior
  truncate code-point-vs-code-unit discussion. Snapshot informing the
  shape of what was looked for.

The cost of maintaining the snapshot (~800 tokens, ~$0.005/run) is
trivially affordable when it materially improves orchestrator triage
on the 1-5 lenses dispatched per review.
2026-05-08 19:28:24 +00:00
Colin McDonnell ae8a634450 action: quieter, deep-linked billing error comments (#600)
* action: quieter, deep-linked billing error comments

The PR progress comment for billing errors led with a loud `### 
Pullfrog billing error` H3 and pointed at the bare `/console` index page
regardless of which org owned the repo. Make the copy quieter and more
actionable:

- bold first line instead of an H3 (the comment already has Pullfrog
  branding in the footer, no need for a second header)
- thread `runContext.repo.owner` into the formatters and deep-link to
  `pullfrog.com/console/<owner>#billing` (or `#model-access` for the
  router-needs-card branch)
- split the old "insufficient balance" default into two branches: card
  declined (Stripe returned a declineCode — "we'll retry next run") vs.
  balance empty (no in-flight charge — "top up or enable auto-reload")
- strip UX framing and pullfrog.com URLs from the proxy-token 402
  responses; they're now terse signal-only strings, with all copy and
  links rendered by the action so there's a single source of truth

* proxy-token: return 503 on phase-1 txn failure, not 402

Phase-1 only fails on server-side issues (serializable retry exhaustion,
Prisma/DB flake) — no Stripe call has happened yet, so it's not a
billing decline. Pre-PR this rendered as the generic "billing error —
manage billing" copy, which was vague-but-not-wrong; under the new
copy it would falsely tell the user their balance is empty.

Returning 503 routes the action through TransientError ("temporarily
unavailable, retry") which is the accurate framing.

Caught by Pullfrog review on PR #600.
2026-05-07 21:40:07 +00:00
Colin McDonnell cd9e00f8d6 test(catalog): refresh latest-model snapshot for google (gemini-3.1-flash-lite) 2026-05-07 21:29:11 +00:00
Colin McDonnell f87e0f878c action: minimize pullfrog.yml permissions and drop actions:read (#594)
* action: minimize pullfrog.yml permissions and drop actions:read

The recommended pullfrog.yml workflow asked for a permissions block that's
broader than what the action actually uses with the workflow GITHUB_TOKEN —
all real work (git push, PR comments, reviews) goes through installation
tokens that the action mints via OIDC. Customer security scanners flagged
the workflow-level block as too permissive.

- Move permissions to the job level and reduce to id-token: write,
  pull-requests: write, issues: write. contents:read is the implicit default
  and covers actions/checkout; contents:write, checks:read are unused by
  any GITHUB_TOKEN consumer; actions:read was only used by post-cleanup's
  listJobsForWorkflowRun call.
- Replace listJobsForWorkflowRun with a SIGTERM/SIGINT handler in main.ts
  that calls core.saveState("cancelled", "true"); post-cleanup reads it
  back via core.getState. Same cancel-vs-failure UX, no extra scope needed.
- Sync the docs (headless-action, getting-started, action/README) and the
  two dogfood pullfrog.yml workflows to the new minimal block. Update the
  post-cleanup wiki to describe the saveState approach.

* action: drop pull-requests/issues from required workflow scopes

Switch postCleanup.ts to mint its own short-lived installation token via OIDC
(acquireNewToken with issues:write + pull_requests:write) instead of using the
workflow GITHUB_TOKEN. Same comment-update behavior, but the workflow no longer
needs those scopes — the only permissions Pullfrog ever asks for are id-token:write
(OIDC exchange) and contents:read (actions/checkout).

Also fixes a bug from the previous commit: setting an explicit permissions block
drops every unlisted scope to none (with metadata as the only exception), so
omitting contents would have broken actions/checkout. Restored at both workflow
and job level.

* action: scope id-token:write to pullfrog job, not workflow level

id-token:write is the powerful one — it lets a job mint OIDC tokens that can
be exchanged for cloud credentials or our installation tokens. Keeping it at
workflow level means any future job added to this file silently inherits it.
Move it to the job level where it's actually used; leave only contents:read
at workflow level as a safe baseline for any future jobs.

* action: move stuck-comment cleanup server-side, drop write perms entirely

The action's post-cleanup step lived inside the runner and used the workflow
GITHUB_TOKEN to update the "Leaping into action…" progress comment when a run
failed/cancelled, requiring pull-requests:write + issues:write at the workflow
level. Move that responsibility to the workflow_run.completed webhook handler:
it already has installation-token access via the GitHub App, runs server-side
(no Pullfrog API dependency loop on failure), and lets us drop both write perms.

Recommended workflow permissions block is now truly minimal:

  permissions:
    contents: read
  jobs:
    pullfrog:
      permissions:
        id-token: write
        contents: read

Server side
- handleWorkflowRunCompleted: when conclusion != "success" and the WorkflowRun
  has progressCommentId, mint installation octokit and update the stuck comment
  in place. Try issues.getComment first, fall back to pulls.getReviewComment on
  404 (we don't store comment type — one wasted GET on the rarer review case).
- Reuses buildPullfrogFooter and updateProgressComment from pullfrog/internal,
  matching the wording the action used to write client-side.

Client side
- Delete action/utils/postCleanup.ts and action/post.ts.
- Remove post: + post-if: from action/action.yml.
- Drop runPostCleanup wiring from action/commands/gha.ts and action/play.ts.
- Remove the SIGTERM/saveState handler I added in main.ts in the previous commit
  (no longer needed; cancel/fail signal comes from the webhook hook payload).

Plumbing
- Extract isLeapingIntoActionCommentBody into action/utils/leapingComment.ts so
  the predicate can be re-exported via pullfrog/internal without dragging the
  MCP server's transitive type graph into the Next.js app's typecheck.
- mcp/comment.ts re-exports from the new location for backward compat.

Wiki
- Delete wiki/post-cleanup.md (obsolete; cleanup is now a one-liner branch in
  the workflow_run webhook handler).

* chore: ignore .worktrees in biome config

Recently-added pnpm worktree feature creates nested git worktrees under
.worktrees/, each with their own biome.jsonc declaring root. Biome's
recursive scan trips on the nested config and fails pnpm lint. Excluding
the directory matches the existing .gitignore entry.

* fix: address PR #594 review findings

Two real bugs caught by code review:

1. handleWorkflowRunWebhook.ts:323 — drop the /m flag on the stuck-comment
   detection regex. With /m, ^ matches any line start, so any finalized
   progress comment that embeds a task list (report_progress writes
   `- [x]`/`- [ ]` lines via todoTracking.ts) would be flagged as "stuck"
   and silently overwritten with the "This run croaked" boilerplate
   whenever the workflow concluded non-success after the agent's final
   summary already landed. Restores the body-start anchoring the original
   in-process postCleanup.ts:90 had.

2. action/scripts/check-entrypoint-imports.ts — drop ../post.ts from the
   esbuild entry-point list (the file was deleted in aa43b9af). The
   `pnpm check:entrypoints` step in test.yml would have failed on every
   run with an unresolvable-entry-point error.

Plus three small follow-ups:
- main.ts:580 — comment said "post-cleanup has its own verify-retry loop"
  but post-cleanup is gone. Updated to describe the new server-side path.
- mcp/comment.ts:443 — comment said "so post script doesn't think the run
  failed". Updated to describe the actual current consumers of wasUpdated.
- commands/gha.ts:84 — `--post` help text said "run post-cleanup flow" but
  with the post-cleanup path removed, --post is only valid alongside the
  `token` subcommand for installation-token revocation. Updated wording.

* fix(action): scope --post help text to gha token subcommand

Root gha help text was documenting --post, but --post only makes sense
paired with the token subcommand (it's how the post step revokes the
installation token previously acquired in the main step). Move it to a
dedicated gha token help section and add a parser layer that rejects
--post on the bare gha command.

  $ pullfrog gha --help
  usage: pullfrog gha [subcommand]
  ...
  options:
    -h, --help   show help

  $ pullfrog gha token --help
  usage: pullfrog gha token [--post]
  ...
  options:
    -h, --help   show help
    --post       revoke the previously-acquired token (post-step usage only)

* webhook: artifact-aware cleanup of stranded leaping comments on success

Previously the workflow_run.completed cleanup only handled non-success
conclusions. Extend it to also catch the rare case where a successful
run leaves a "Leaping into action…" comment stuck (in-process cleanup at
action/main.ts:723 normally handles this, but can be skipped on SIGKILL,
runner host crash, or any exit path that bypasses main()'s finally block).

New behavior in cleanupStuckProgressComment:

  - cancelled       → update with "cancelled 🛑" body  (unchanged)
  - failure (other) → update with "croaked 😵" body    (unchanged)
  - success + artifact recorded → delete the comment (the artifact is the
                                  user-facing surface; the leaping comment
                                  is just stale UI noise at this point)
  - success + no artifact recorded → delete the comment AND alert
                                     team@pullfrog.com via emailAlert

The "success + no artifact" path is "should never happen" territory: the
run claims success but produced no review, PR, issue, plan, or summary
comment. The team alert helps us catch in-process cleanup regressions or
artifact-tracking gaps. hasRecordedArtifact reads {review,pr,issue,
planComment,summaryComment}NodeId off the WorkflowRun row to make the call.

* webhook: narrow stuck-comment detection to leaping prefix only

Drop the stranded-todo-pattern branch from cleanupStuckProgressComment.
The leaping prefix is highly specific and impossible to confuse with a
legitimate summary; a leading todo line is not — the agent's
error-reporting paths can produce useful explanatory comments whose
body leads with a checklist (e.g. "here's what I was working on" + the
incomplete todo list), and we don't want to silently overwrite those
with the generic "croaked" boilerplate.

In-process cleanup at action/main.ts:723 still handles the stranded-todo
case in the common path (gated on !finalSummaryWritten with full access
to the in-memory tool state). Missing the rare runner-died-mid-todo case
server-side is a worthwhile trade vs. the false-positive risk on real
explanatory comments.
2026-05-07 18:59:52 +00:00
Colin McDonnell e2e29a19fc accept pullfrog.yaml as well as pullfrog.yml (#596)
* accept pullfrog.yaml as well as pullfrog.yml

centralize the accepted workflow filenames in `utils/github/pullfrogWorkflow.ts`
(`PULLFROG_WORKFLOW_FILES = ["pullfrog.yml", "pullfrog.yaml"]`) and use the new
`findExistingWorkflowFile` helper at every read path: `getWorkflow` (cached),
the verify-workflow API route, and the audit/sync/download/update scripts. `.yml`
is always tried first so the common case still costs exactly one API call.

webhook handlers (push cache-bust, `workflow_run_*`) now use the shared
`isPullfrogWorkflowPath` matcher.

action runtime (`reviewCleanup.ts`) derives the running workflow's filename from
`process.env.GITHUB_WORKFLOW_REF` instead of hardcoding `.yml`, so the safety-net
follow-up dispatch targets whichever file the user actually has — strictly more
correct than today.

write paths (`createWorkflowForRepo`, `createWorkflowPR`) intentionally still
create `.yml`; existing 422 collision handling covers the rare double-install
case. UI/wiki/onboarding copy keeps saying `pullfrog.yml`; one callout in
`docs/getting-started.mdx` mentions `.yaml` works too.

also drops dead code (`utils/github/findWorkflow.ts`, parallel single-file
implementation with no importers) and the now-unused `WORKFLOW_FILENAME` export.

* rename pullfrogWorkflow.ts -> findPullfrogWorkflow.ts (verb form)

* add pre-flight check to workflow create paths

`createWorkflowForRepo` and `createWorkflowPR` now check for any existing
pullfrog workflow file (`.yml` or `.yaml`) before doing work, preventing the
degenerate state where a repo with `pullfrog.yaml` ends up with both files
dispatching on every event.

costs one `getContent` call per first-time install. existing 422 branch in
`createWorkflowForRepo` is retained as a race-condition safety net; the 409
branch now also handles the case where `createWorkflowPR` discovers an
existing file in flight.

`createWorkflowPR` return shape becomes a discriminated union; the standalone
`/api/create-workflow-pr` route returns `{ alreadyInstalled: true }` instead
of creating a redundant PR.

* promote repo to active when /api/create-workflow-pr finds existing workflow

extracts `promoteRepoToActive` from `createWorkflowForRepo`'s closure to a
shared module-level function, and wires it into the standalone PR route's
`alreadyInstalled` branch so a `needs_setup` repo with an existing `.yaml`
file doesn't go stale (was only handled by the dashboard's own create path).

addresses pullfrog review on #596.
2026-05-07 18:04:07 +00:00
pullfrog[bot] 6f76a6a9da fix(action): tighten provider error detection and propagate agent error events (#580)
* fix(action): tighten provider error detection and propagate agent error events

Both bugs from #562:

1. detectProviderError used substring matches against "429", "rate limit",
   etc. — false-positives on commit SHAs containing 429 and on x-ratelimit-*
   response headers in dumped 401 error JSON. rewrote with anchored regexes:
   numeric status codes only match adjacent to a recognised status key, and
   `\brate[_ ]limit(?=[_ ]|\b)` rejects ratelimit-* headers (no separator).
   word-boundary anchors on INTERNAL / UNAVAILABLE / quota / limit:0 reject
   INTERNAL_SERVER_ERROR / time_limit:0 substrings. added 11-case regression
   test.

2. opencode 401s slipped through `eventCount === 0 && lastProviderError`
   because opencode's own type=error event increments eventCount before
   the guard runs. added an explicit `error:` handler that captures the
   event and propagates it to a non-success AgentResult. opencode emits
   the message under `error.data.message`, not the top level. mirror fix
   in claude.ts: error_max_turns / error_during_execution / any error*
   subtype on the result event now flips success: false.

* fix(action): match quota inside identifiers like insufficient_quota

\bquota\b missed insufficient_quota / quota_exceeded / quotaExceeded
because _ is a word character and camelCase has no boundary. quota is
specific enough to be matched as a plain substring.

* fix(action): match `rate limited` and `rate limits exceeded`

Drop the trailing `(?=[_ ]|\b)` lookahead from the rate-limit regex. The
lookahead failed when `limit` was followed by another word character
(`limited`, `limits`), so `rate limited` and `rate limits exceeded` were
slipping past detection. The leading `\b` plus `[_ ]` separator already
rejects `x-ratelimit-*` / `anthropic-ratelimit-*` headers without it.

---------

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: David Blass <david@arktype.io>
2026-05-07 16:31:05 +00:00
pullfrog[bot] 366af55f19 fix(action): sweep stale .git/*.lock and deepen-retry shallow git_fetch (#564) (#578)
- checkoutPrBranch now removes .git/shallow.lock, .git/index.lock, and
  .git/objects/maintenance.lock when older than 30s before the first fetch.
  prior runs that crashed mid-fetch left these behind on self-hosted runners,
  causing checkout_pr to abort with `Unable to create '.git/shallow.lock':
  File exists` until the agent shelled out to rm -f.

- GitFetchTool catches `Could not read <sha>` and `remote did not send all
  necessary objects` on shallow clones and retries once with --deepen=1000
  instead of bouncing the failure back to the agent. agents previously had
  to fall back to checking out FETCH_HEAD, losing branch context.

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: David Blass <david@arktype.io>
2026-05-06 22:38:41 +00:00
pullfrog[bot] 4c1413d925 fix(action): flip wasUpdated on substantive MCP write tools (#563) (#577)
* fix(action): flip wasUpdated on substantive MCP write tools (#563)

Review/Respond/etc. agents that submit a `create_pull_request_review`,
`create_issue_comment`, or `update_pull_request_body` and exit without
calling `report_progress` were being marked as workflow failures by the
strict completion check in handleAgentResult. Extend the set of tools
that flip toolState.wasUpdated so a substantive user-visible artifact
satisfies the check. The isReviewMode bypass is retained for
IncrementalReview's non-substantive path.

Flag is set BEFORE patchWorkflowRunFields / deleteProgressComment in
each tool so a best-effort cleanup failure does not undo the signal.

* fix(action): use finalSummaryWritten for stranded progress cleanup

The stranded-progress-comment cleanup at the end of main() previously
fired only when toolState.wasUpdated was false (or the tracker was the
last writer). With wasUpdated now set by additional MCP write tools
(create_issue_comment, update_pull_request_body), an agent that produced
a substantive artifact via one of those tools and skipped report_progress
would leave the placeholder "Leaping into action" comment intact — the
post-script then converted it into an error message on a successful run.

Key the cleanup off finalSummaryWritten instead. That flag is only set
when report_progress actually wrote the progress comment, so it cleanly
distinguishes "comment is finalized" from "agent did other work but
never touched the progress comment".

* refactor(mcp): extract markSubstantiveArtifact() helper

replaces 4 inline `ctx.toolState.wasUpdated = true` flips in CreateCommentTool, UpdatePullRequestBodyTool, and CreatePullRequestReviewTool with a single helper in mcp/server.ts. JSDoc on the helper documents the contract (call BEFORE downstream patch/cleanup; gates the strict completion check and stranded-comment cleanup) so future MCP write tool authors only need to grep for one symbol.

no behavioral change.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): only flip finalSummaryWritten after non-skipped write

Previously the flag was set unconditionally on any non-plan call,
including paths where reportProgress skipped (silent events, deleted
comment, no issue/PR target). The cleanup check in main.ts is
safeguarded by toolState.progressComment so the bug doesn't manifest
today, but aligning the flag with actual writes matches the wasUpdated
pattern and the design intent in the cleanup plan.

* refactor(mcp): inline markSubstantiveArtifact helper

---------

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: David Blass <david@arktype.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-06 21:05:53 +00:00
pullfrog[bot] 6db4a6d02e fix(mcp): preserve coveragePreflightRan across checkout_pr refreshes (#576)
checkout_pr unconditionally rebuilds ctx.toolState.diffCoverage via
createDiffCoverageState, which initialised coveragePreflightRan to false.
a second checkout_pr therefore reset the "one-time nudge per review
session" guarantee in runDiffCoveragePreflight, and the next
create_pull_request_review threw the diff-coverage pre-flight error
again — even after the agent had already gone through the
read-and-resubmit dance once.

createDiffCoverageState now accepts an optional previous state and
carries forward coveragePreflightRan. coveredRanges are intentionally
not carried because their line numbers are tied to the previous diff's
content (especially under incremental diffs).

closes #566

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: David Blass <david@arktype.io>
2026-05-06 20:54:34 +00:00
Colin McDonnell 3c8b493aee modes: soften "two-out-of-three" rule from veto to look-harder signal
The previous phrasing ("not enough — still degrades the codebase") read as a
categorical claim that elegance vetoes correctness, which inverts the usual
hierarchy and risks giving the agent a clean rationalization for rejecting
genuine correctness fixes. Reframe as a prompt to keep searching for a fix
that gets all three before accepting the trade — preserves the pressure
without the absolute.
2026-05-06 03:01:06 +00:00
41 changed files with 1253 additions and 569 deletions
+4 -6
View File
@@ -12,16 +12,14 @@ on:
description: Run name
permissions:
id-token: write
contents: write
pull-requests: write
issues: write
actions: read
checks: read
contents: read
jobs:
pullfrog:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v6
+5 -11
View File
@@ -72,16 +72,14 @@ on:
description: 'Agent prompt'
permissions:
id-token: write
contents: write
pull-requests: write
issues: write
actions: read
checks: read
contents: read
jobs:
pullfrog:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v6
@@ -130,11 +128,7 @@ jobs:
permissions:
id-token: write
contents: write
issues: write
pull-requests: write
actions: read
checks: read
contents: read
uses: ./.github/workflows/pullfrog.yml
with:
# pass the full event payload as the prompt
-2
View File
@@ -36,8 +36,6 @@ outputs:
runs:
using: "node24"
main: "entry.ts"
post: "post.ts"
post-if: "failure() || cancelled()"
branding:
icon: "code"
+24
View File
@@ -210,6 +210,7 @@ async function runClaude(params: RunParams): Promise<ClaudeRunResult> {
let finalOutput = "";
let sessionId: string | undefined;
let resultErrorSubtype: string | null = null;
let accumulatedTokens = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
// Claude CLI reports a single end-of-run `total_cost_usd` on the result
// event. per-message events don't carry cost, so there's nothing to sum —
@@ -367,9 +368,14 @@ async function runClaude(params: RunParams): Promise<ClaudeRunResult> {
tokensLogged = true;
}
} else if (subtype === "error_max_turns") {
resultErrorSubtype = subtype;
log.info(`» ${params.label} max turns reached: ${JSON.stringify(event)}`);
} else if (subtype === "error_during_execution") {
resultErrorSubtype = subtype;
log.info(`» ${params.label} execution error: ${JSON.stringify(event)}`);
} else if (subtype.startsWith("error")) {
resultErrorSubtype = subtype;
log.info(`» ${params.label} result: subtype=${subtype}, data=${JSON.stringify(event)}`);
} else {
log.info(`» ${params.label} result: subtype=${subtype}, data=${JSON.stringify(event)}`);
}
@@ -401,6 +407,12 @@ async function runClaude(params: RunParams): Promise<ClaudeRunResult> {
activityTimeout: 300_000,
onActivityTimeout: params.onActivityTimeout,
stdio: ["ignore", "pipe", "pipe"],
// run claude in its own process group so SIGKILL on activity timeout /
// outer cancellation reaches any subprocesses it spawns (rg, file
// watchers, mcp transports, etc). claude itself is a node bundle so
// there's no shim-orphan issue like opencode-ai/bin/opencode, but
// detached + killGroup is the right default for any agent runtime.
killGroup: true,
onStdout: async (chunk) => {
const text = chunk.toString();
output += text;
@@ -527,6 +539,16 @@ async function runClaude(params: RunParams): Promise<ClaudeRunResult> {
};
}
if (resultErrorSubtype) {
return {
success: false,
output: finalOutput || output,
error: `result subtype: ${resultErrorSubtype}`,
usage,
sessionId,
};
}
return { success: true, output: finalOutput || output, usage, sessionId };
} catch (error) {
params.todoTracker?.cancel();
@@ -703,6 +725,8 @@ export const claude = agent({
initialResult: result,
initialUsage: result.usage,
stopScript: ctx.stopScript,
summaryFilePath: ctx.summaryFilePath,
summarySeed: ctx.summarySeed,
reflectionPrompt: buildLearningsReflectionPrompt("claude"),
canResume: (r) => Boolean(r.sessionId),
resume: async (c) => {
+74 -1
View File
@@ -59,6 +59,22 @@ type OpenCodeConfig = {
[key: string]: unknown;
};
/**
* Per-inference `max_tokens` reservation the agent sends to the upstream
* model. OpenCode's default is 32_000 (sized for long-running TUI sessions
* where a human user might want big outputs). Pullfrog runs are headless and
* short — typical outputs are 1-3K tokens — so we cap at 5_000. This
* drastically reduces the upfront budget reservation OpenRouter requires per
* call (~$0.38 vs ~$2.40 for Opus), which is what lets low-wallet runs
* actually start.
*
* Plumbed via `OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX` env var rather than the
* config JSON. OpenCode's `OUTPUT_TOKEN_MAX` (session/llm.ts) is sourced
* exclusively from this env var; top-level `limit.output` in the config
* has no read site and is silently dropped on merge.
*/
const PULLFROG_OPENCODE_OUTPUT_LIMIT = 5000;
function buildSecurityConfig(ctx: AgentRunContext, model: string | undefined): string {
const config: OpenCodeConfig = {
permission: {
@@ -254,7 +270,13 @@ interface OpenCodeErrorEvent {
type: "error";
timestamp?: string;
sessionID?: string;
error?: { name?: string; message?: string; data?: unknown; [key: string]: unknown };
// opencode emits the error message under `error.data.message`, not at the
// top level. see anomalyco/opencode packages/opencode/src/cli/cmd/run.ts.
error?: {
name?: string;
data?: { message?: string; [key: string]: unknown };
[key: string]: unknown;
};
[key: string]: unknown;
}
@@ -346,6 +368,16 @@ async function runOpenCode(params: RunParams): Promise<AgentResult> {
// mismatched callID" signal).
const knownNonTaskCallIDs = new Set<string>();
// a subagent is "in flight" while at least one `task` tool_use is awaiting
// its tool_result. used to suspend spawn's activity timer (which only sees
// opencode's stdout) so a long-running subagent — whose internal events
// don't surface on the parent NDJSON stream — can't trip the 5min timeout.
// boolean state instead of a heartbeat so there's no race window between a
// periodic tick and a subagent that finishes between ticks.
function isSubagentInFlight(): boolean {
return taskDispatchByCallID.size > 0 || pendingTaskDispatches.length > 0;
}
function emitSubagentFinished(
dispatch: TaskDispatch,
status: string,
@@ -630,6 +662,16 @@ async function runOpenCode(params: RunParams): Promise<AgentResult> {
log.debug(withLabel(label, `tool output: ${outputStr}`));
}
},
error: (event: OpenCodeErrorEvent) => {
// opencode emits a `type=error` event when a provider call fails (e.g.
// 401 Invalid authentication credentials). the underlying CLI still
// exits 0 because the error was returned cleanly by the LLM SDK, so
// unless we capture this event the run is reported as success.
agentErrorEvent = event;
const errorName = event.error?.name || "unknown";
const errorMessage = event.error?.data?.message || event.error?.name || JSON.stringify(event);
log.info(`» ${params.label} error event: ${errorName}: ${errorMessage}`);
},
result: async (event: OpenCodeResultEvent) => {
const status = event.status || "unknown";
const duration = event.stats?.duration_ms || 0;
@@ -663,6 +705,7 @@ async function runOpenCode(params: RunParams): Promise<AgentResult> {
const recentStderr: string[] = [];
let lastProviderError: string | null = null;
let agentErrorEvent: OpenCodeErrorEvent | null = null;
let output = "";
let stdoutBuffer = "";
@@ -676,6 +719,20 @@ async function runOpenCode(params: RunParams): Promise<AgentResult> {
activityTimeout: 300_000,
onActivityTimeout: params.onActivityTimeout,
stdio: ["ignore", "pipe", "pipe"],
// node_modules/opencode-ai/bin/opencode is a Node shim that spawnSyncs
// the native opencode-<plat>-<arch> binary with stdio:"inherit". without
// a process-group kill, SIGKILL hits only the shim, the native binary
// is reparented to PID 1, holds our stdout pipe open, and `child.close`
// never fires — producing zombie runs. detached + killGroup nukes the
// whole tree.
killGroup: true,
// suspend the inner activity timer while a `task` subagent is in flight.
// opencode's task tool encapsulates subagent execution in-process — the
// subagent's internal events don't surface on the parent NDJSON stream,
// so without this the 5min timeout would falsely fire mid-subagent.
// suspend/resume is preferable to a heartbeat because there's no race
// between a periodic tick and a subagent finishing between ticks.
isPausedExternally: isSubagentInFlight,
onStdout: async (chunk) => {
const text = chunk.toString();
output += text;
@@ -824,6 +881,19 @@ async function runOpenCode(params: RunParams): Promise<AgentResult> {
};
}
if (agentErrorEvent) {
const errorEvent: OpenCodeErrorEvent = agentErrorEvent;
const errorName = errorEvent.error?.name || "agent error";
const errorMessage =
errorEvent.error?.data?.message || errorEvent.error?.name || JSON.stringify(errorEvent);
return {
success: false,
output: finalOutput || output,
error: `${errorName}: ${errorMessage}`,
usage,
};
}
return { success: true, output: finalOutput || output, usage };
} catch (error) {
params.todoTracker?.cancel();
@@ -899,6 +969,7 @@ export const opencode = agent({
...homeEnv,
OPENCODE_CONFIG_CONTENT: buildSecurityConfig(ctx, model),
OPENCODE_PERMISSION: permissionOverride,
OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX: PULLFROG_OPENCODE_OUTPUT_LIMIT.toString(),
GOOGLE_GENERATIVE_AI_API_KEY:
process.env.GOOGLE_GENERATIVE_AI_API_KEY || process.env.GEMINI_API_KEY,
};
@@ -932,6 +1003,8 @@ export const opencode = agent({
initialResult: result,
initialUsage: result.usage,
stopScript: ctx.stopScript,
summaryFilePath: ctx.summaryFilePath,
summarySeed: ctx.summarySeed,
reflectionPrompt: buildLearningsReflectionPrompt("opencode"),
resume: async (c) =>
runOpenCode({
+82 -3
View File
@@ -1,3 +1,4 @@
import { readFile } from "node:fs/promises";
import { type AgentId, formatMcpToolRef } from "../external.ts";
import { LIFECYCLE_HOOK_TIMEOUT_MS } from "../lifecycle.ts";
import { log } from "../utils/cli.ts";
@@ -92,13 +93,46 @@ export function buildStopHookPrompt(failure: StopHookFailure): string {
].join("\n");
}
/** check whether the seeded summary file is byte-identical to its seed.
* a missing or unreadable file returns false (don't nudge — the agent
* may have legitimately deleted it, or the seed step failed; the read-
* back path in main.ts handles both cases by skipping persist). */
async function isSummaryUnchanged(filePath: string, seed: string): Promise<boolean> {
try {
const current = await readFile(filePath, "utf8");
return current === seed;
} catch {
return false;
}
}
export function buildSummaryStalePrompt(filePath: string): string {
return [
`PR SUMMARY UNTOUCHED — the rolling PR summary file at \`${filePath}\` is byte-identical to its seed; this run did not edit it.`,
"",
"review the diff and update the file in place to reflect what changed in the PR. update intent, key changes, and any risks worth flagging — keep the existing section headings stable so incremental runs produce clean diffs.",
"",
"if the diff is genuinely too small or noisy to warrant rewriting (e.g. a one-line typo fix, a comment tweak, a formatting-only change), it's fine to leave the structure as-is — but at minimum confirm you considered it by appending one line to the appropriate section noting the run. silence is not an option; the snapshot is what the next review run reads as context.",
].join("\n");
}
/**
* check the two post-run gates: did the stop hook pass and is the working
* tree clean? returns everything that still needs fixing so the caller can
* check the post-run gates: did the stop hook pass, is the working tree
* clean, and (when applicable) did the agent touch the rolling PR summary
* snapshot? returns everything that still needs nudging so the caller can
* render a single combined resume prompt.
*
* the summary-stale check is skipped when `summaryFilePath` / `summarySeed`
* are not provided; this is the common case (non-PR runs, runs where the
* dispatcher didn't request snapshot generation, runs where the seed step
* failed). loop callers also pass these as undefined after the agent has
* already been nudged once, to avoid burning the retry budget on a soft
* non-blocking gate.
*/
export async function collectPostRunIssues(params: {
stopScript: string | null | undefined;
summaryFilePath?: string | undefined;
summarySeed?: string | undefined;
}): Promise<PostRunIssues> {
const issues: PostRunIssues = {};
if (params.stopScript) {
@@ -107,6 +141,10 @@ export async function collectPostRunIssues(params: {
}
const status = getGitStatus();
if (status) issues.dirtyTree = status;
if (params.summaryFilePath && params.summarySeed !== undefined) {
const stale = await isSummaryUnchanged(params.summaryFilePath, params.summarySeed);
if (stale) issues.summaryStale = { filePath: params.summaryFilePath };
}
return issues;
}
@@ -114,6 +152,7 @@ export function buildPostRunPrompt(issues: PostRunIssues): string {
const parts: string[] = [];
if (issues.stopHook) parts.push(buildStopHookPrompt(issues.stopHook));
if (issues.dirtyTree) parts.push(buildCommitPrompt(issues.dirtyTree));
if (issues.summaryStale) parts.push(buildSummaryStalePrompt(issues.summaryStale.filePath));
return parts.join("\n\n---\n\n");
}
@@ -164,6 +203,14 @@ export async function runPostRunRetryLoop<R extends AgentResult>(params: {
initialResult: R;
initialUsage: AgentUsage | undefined;
stopScript: string | null | undefined;
/** absolute path to the seeded PR summary file. when set together with
* `summarySeed`, the loop checks after each agent attempt whether the
* file has been edited; if not, it nudges the agent ONCE via a resume
* turn (subsequent iterations skip the check so we don't keep burning
* retries on a soft gate when the agent has decided no edit is warranted). */
summaryFilePath?: string | undefined;
/** exact bytes of the seeded summary file used for the unchanged-check. */
summarySeed?: string | undefined;
resume: (context: { prompt: string; previousResult: R }) => Promise<R>;
canResume?: ((result: R) => boolean) | undefined;
reflectionPrompt?: string | undefined;
@@ -173,10 +220,21 @@ export async function runPostRunRetryLoop<R extends AgentResult>(params: {
let finalIssues: PostRunIssues = {};
let gateResumeCount = 0;
let pendingReflection = params.reflectionPrompt;
// nudge for an untouched summary file fires AT MOST ONCE per run. after
// we've delivered the prompt, subsequent gate checks pass undefined so
// the loop doesn't keep flagging the same condition — the agent may have
// legitimately decided no edit is warranted, and re-prompting would
// burn the retry budget without adding signal.
let summaryStaleNudged = false;
while (gateResumeCount < MAX_POST_RUN_RETRIES) {
if (!result.success) break;
const issues = await collectPostRunIssues({ stopScript: params.stopScript });
const issues = await collectPostRunIssues({
stopScript: params.stopScript,
summaryFilePath: summaryStaleNudged ? undefined : params.summaryFilePath,
summarySeed: summaryStaleNudged ? undefined : params.summarySeed,
});
if (issues.summaryStale) summaryStaleNudged = true;
finalIssues = issues;
if (!hasPostRunIssues(issues)) {
@@ -230,8 +288,25 @@ export async function runPostRunRetryLoop<R extends AgentResult>(params: {
log.info(`» post-run retry (attempt ${gateResumeCount + 1}/${MAX_POST_RUN_RETRIES})`);
const prompt = buildPostRunPrompt(issues);
// summary-stale is a soft gate that must never flip a successful run to
// failed. when it's the only issue and the resume itself errors out,
// restore the pre-resume successful result and break — persistSummary
// detects the unchanged file via its seed comparison and skips the DB
// write on its own, so no further coordination is needed here.
const onlySummaryStale =
issues.summaryStale !== undefined &&
issues.stopHook === undefined &&
issues.dirtyTree === undefined;
const preResume = result;
result = await params.resume({ prompt, previousResult: result });
aggregatedUsage = mergeAgentUsage(aggregatedUsage, result.usage);
if (!result.success && onlySummaryStale) {
log.warning(
`» summary-stale resume turn failed (${result.error ?? "unknown error"}), preserving prior successful result`
);
result = preResume;
break;
}
gateResumeCount++;
}
@@ -242,6 +317,10 @@ export async function runPostRunRetryLoop<R extends AgentResult>(params: {
// already observed a clean state we skip: re-running the hook risks flaky
// false-positive failures right after it just passed.
if (gateResumeCount > 0 && result.success && hasPostRunIssues(finalIssues)) {
// re-check the gates that can actually fail the run (stop hook /
// dirty tree). summary-stale is intentionally NOT re-checked here:
// we already delivered the one-shot nudge, and a still-unchanged
// file at this point is the agent's deliberate choice.
finalIssues = await collectPostRunIssues({ stopScript: params.stopScript });
}
+28 -1
View File
@@ -42,13 +42,26 @@ export interface StopHookFailure {
output: string;
}
export interface SummaryStale {
/** absolute path to the seeded snapshot file the agent was meant to edit. */
filePath: string;
}
export interface PostRunIssues {
stopHook?: StopHookFailure;
dirtyTree?: string;
/** populated when the rolling PR summary file is byte-identical to its
* seed, i.e. the agent never touched it. soft gate — nudges once via a
* resume turn but never fails the run, parallel to dirtyTree semantics. */
summaryStale?: SummaryStale;
}
export function hasPostRunIssues(issues: PostRunIssues): boolean {
return issues.stopHook !== undefined || issues.dirtyTree !== undefined;
return (
issues.stopHook !== undefined ||
issues.dirtyTree !== undefined ||
issues.summaryStale !== undefined
);
}
/**
@@ -106,6 +119,20 @@ export interface AgentRunContext {
* guidance. null when the repo has no stop hook configured.
*/
stopScript?: string | null | undefined;
/**
* absolute path to the rolling PR summary tmpfile, when one was seeded
* for this run (Review / IncrementalReview / pr-summary Task). enables
* a post-run sanity nudge that prompts the agent if the file is still
* byte-identical to its seed.
*/
summaryFilePath?: string | undefined;
/**
* exact bytes of the seeded summary file. compared against the current
* file content after each agent attempt to detect "agent forgot to edit
* the summary" — particularly common with smaller models that lose
* track of multi-step instructions.
*/
summarySeed?: string | undefined;
/**
* called synchronously when the agent subprocess is killed for inner
* activity timeout. lets main.ts tear down shared resources (MCP HTTP
+52 -26
View File
@@ -2,8 +2,6 @@ import { dirname } from "node:path";
import * as core from "@actions/core";
import arg from "arg";
import { main } from "../main.ts";
import { log } from "../utils/cli.ts";
import { runPostCleanup } from "../utils/postCleanup.ts";
import { acquireInstallationToken, revokeInstallationToken } from "../utils/token.ts";
// GitHub Actions runs the action entry point with the node24 binary specified
@@ -31,16 +29,6 @@ async function runMain(): Promise<void> {
}
}
async function runPost(): Promise<void> {
log.debug(`[post] script started at ${new Date().toISOString()}`);
try {
await runPostCleanup();
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
log.error(`[post] unexpected error: ${message}`);
}
}
async function tokenMain(): Promise<void> {
const reposInput = core.getInput("repos");
const additionalRepos = reposInput
@@ -73,7 +61,7 @@ async function tokenPost(): Promise<void> {
}
function printGhaUsage(params: { stream: typeof console.log; prog: string }): void {
params.stream(`usage: ${params.prog} gha [token] [--post]\n`);
params.stream(`usage: ${params.prog} gha [subcommand]\n`);
params.stream("run the github action runtime flow.");
params.stream("");
params.stream("subcommands:");
@@ -81,10 +69,31 @@ function printGhaUsage(params: { stream: typeof console.log; prog: string }): vo
params.stream("");
params.stream("options:");
params.stream(" -h, --help show help");
params.stream(" --post run post-cleanup flow");
}
function printGhaTokenUsage(params: { stream: typeof console.log; prog: string }): void {
params.stream(`usage: ${params.prog} gha token [--post]\n`);
params.stream("acquire a github app installation token, or revoke it in the post step.");
params.stream("");
params.stream("options:");
params.stream(" -h, --help show help");
params.stream(" --post revoke the previously-acquired token (post-step usage only)");
}
function parseGhaArgs(args: string[]) {
return arg(
{
"--help": Boolean,
"-h": "--help",
},
{
argv: args,
stopAtPositional: true,
}
);
}
function parseGhaTokenArgs(args: string[]) {
return arg(
{
"--help": Boolean,
@@ -118,27 +127,46 @@ export async function runCli(params: GhaCliParams): Promise<void> {
return;
}
const normalizedArgs = ["gha"];
const positional = parsed._;
const subcommand = positional[0];
if (positional.length > 1) {
console.error(`unexpected positional arguments for gha: ${positional.slice(1).join(" ")}\n`);
if (!subcommand) {
await run(["gha"]);
return;
}
if (subcommand !== "token") {
console.error(`unknown gha subcommand: ${subcommand}\n`);
printGhaUsage({ stream: console.error, prog: params.prog });
process.exit(1);
}
if (positional[0] === "token") {
normalizedArgs.push("token");
} else if (positional[0]) {
console.error(`unknown gha subcommand: ${positional[0]}\n`);
printGhaUsage({ stream: console.error, prog: params.prog });
// gha token [--post]
let tokenParsed: ReturnType<typeof parseGhaTokenArgs>;
try {
tokenParsed = parseGhaTokenArgs(positional.slice(1));
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(`${message}\n`);
printGhaTokenUsage({ stream: console.error, prog: params.prog });
process.exit(1);
}
if (parsed["--post"]) {
if (tokenParsed["--help"]) {
printGhaTokenUsage({ stream: console.log, prog: params.prog });
return;
}
if (tokenParsed._.length > 0) {
console.error(`unexpected positional arguments for gha token: ${tokenParsed._.join(" ")}\n`);
printGhaTokenUsage({ stream: console.error, prog: params.prog });
process.exit(1);
}
const normalizedArgs = ["gha", "token"];
if (tokenParsed["--post"]) {
normalizedArgs.push("--post");
}
await run(normalizedArgs);
}
@@ -150,8 +178,6 @@ export async function run(args: string[]) {
} else {
await tokenMain();
}
} else if (args.includes("--post")) {
await runPost();
} else {
await runMain();
}
+2
View File
@@ -281,6 +281,8 @@ export interface WriteablePayload {
cwd?: string | undefined;
/** pre-created progress comment (ID + type) for updating status */
progressComment?: { id: string; type: "issue" | "review" } | undefined;
/** when true, seed the PR summary tmpfile + persist edits at run end */
generateSummary?: boolean | undefined;
}
// immutable payload type for agent execution
+4
View File
@@ -40,6 +40,10 @@ export {
stripExistingFooter,
} from "../utils/buildPullfrogFooter.ts";
export type { ResourceUsage, UsageSummary } from "../utils/github.ts";
export {
isLeapingIntoActionCommentBody,
LEAPING_INTO_ACTION_PREFIX,
} from "../utils/leapingComment.ts";
export type {
CreateProgressCommentTarget,
ProgressComment,
+248 -47
View File
@@ -1,6 +1,7 @@
// changes to tool permissions should be reflected in wiki/granular-tools.md
import { existsSync, readdirSync } from "node:fs";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import * as core from "@actions/core";
import { deleteProgressComment, reportProgress } from "./mcp/comment.ts";
@@ -34,6 +35,8 @@ import { executeLifecycleHook } from "./utils/lifecycle.ts";
import { normalizeEnv } from "./utils/normalizeEnv.ts";
import { aggregateUsage, patchWorkflowRunFields } from "./utils/patchWorkflowRunFields.ts";
import { resolvePayload, resolvePromptInput } from "./utils/payload.ts";
import { isRouterKeylimitExhaustedError } from "./utils/providerErrors.ts";
import { readSummaryFile, seedSummaryFile } from "./utils/prSummary.ts";
import { postReviewCleanup } from "./utils/reviewCleanup.ts";
import { handleAgentResult } from "./utils/run.ts";
import { type AccountPlan, isInfraCovered } from "./utils/runContext.ts";
@@ -160,55 +163,119 @@ class TransientError extends Error {
}
/**
* Render a BillingError as user-facing markdown (shared between GH job summary
* and the PR progress comment). Branches:
*
* - `router_requires_card`: the user is on Router mode with no card AND
* no wallet balance. Points at the add-card flow in the console.
* - `needsReauthentication` (Stripe `authentication_required` decline): the
* issuer requires a 3D Secure challenge on each off-session charge —
* re-adding the card won't help because the issuer's policy persists
* across credentials. The escape valve is a manual top-up from the
* dashboard, where 3DS runs interactively inside Stripe Checkout.
* - default: generic "manage billing" with the declineCode appended if
* classified (insufficient_funds, lost_card, generic_decline, etc).
* Deep link into the right console section for the failing account. Anchors
* are defined in `app/console/[owner]/page.tsx` (`#billing`, `#model-access`).
* `owner` is the GitHub login of the repo's account — i.e. the org or user
* that pays for this repo's runs, which is the right scope for billing.
*/
function formatBillingErrorSummary(error: BillingError): string {
function billingConsoleUrl(owner: string, anchor: "billing" | "model-access"): string {
return `https://pullfrog.com/console/${encodeURIComponent(owner)}#${anchor}`;
}
/**
* Render a BillingError as user-facing markdown (shared between GH job summary
* and the PR progress comment). Goals:
*
* - quiet, not alarmist — bold first line instead of an `### ❌` H3, since
* the comment already has Pullfrog branding in the footer
* - actionable — every branch ends in a single CTA deep-linked to the
* correct section of the owner's console
* - honest — say what actually went wrong (card declined vs. balance
* empty vs. 3DS required), don't lump them under "billing error"
*
* Branches:
* - `router_requires_card`: user is on Router mode with no card AND no
* wallet balance. Lead with the carrot ($20 free credit), link to
* `#model-access` where the Add Card flow lives.
* - `router_balance_exhausted`: user has a card on file but auto-reload is
* disabled and they've spent past their $5 overdraft buffer. Frame as
* "balance ran out" and surface both remediation paths (top up, or flip
* on auto-reload).
* - `router_keylimit_exhausted`: OpenRouter rejected mid-run because the
* per-run key budget was exhausted while the agent was working. The
* wallet is now negative; same remediation as `router_balance_exhausted`
* but framed for the after-the-fact case ("this run was cut short").
* - `needsReauthentication`: issuer requires 3DS on every off-session
* charge. Re-adding the card won't help — the only escape is a manual
* top-up where 3DS runs interactively in Stripe Checkout.
* - `declineCode` set: Stripe declined a real charge. Show the sub-code
* so support can act on it; tell the user we'll retry on next dispatch.
* - default: balance hit zero with no in-flight charge (auto-reload off
* or amount below threshold). Direct them to top up or enable auto-reload.
*/
function formatBillingErrorSummary(error: BillingError, owner: string): string {
if (error.code === "router_requires_card") {
return [
"### ⛔ Pullfrog Router requires a card",
"**Add a card to start using Pullfrog Router.**",
"",
"This run was going to use Pullfrog Router, which bills at raw OpenRouter cost and needs a card on file. Runs won't proceed until a card is added.",
"Router proxies OpenRouter at raw cost — no platform markup, and your first $20 of usage is on us.",
"",
"[Add a card →](https://pullfrog.com/console#model-access) — your first $20 of Router usage is free.",
`[Add a card →](${billingConsoleUrl(owner, "model-access")})`,
].join("\n");
}
if (error.code === "router_balance_exhausted") {
return [
"**Your Pullfrog Router balance is exhausted.**",
"",
"You have a card on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.",
"",
`[Top up balance →](${billingConsoleUrl(owner, "billing")}) · [Enable auto-reload →](${billingConsoleUrl(owner, "model-access")})`,
].join("\n");
}
if (error.code === "router_keylimit_exhausted") {
return [
"**This run was cut short — your Pullfrog Router balance ran out mid-run.**",
"",
"OpenRouter stopped the agent because the per-run budget was exhausted. Your wallet is now negative; top up or enable auto-reload to keep runs flowing.",
"",
`[Top up balance →](${billingConsoleUrl(owner, "billing")}) · [Enable auto-reload →](${billingConsoleUrl(owner, "model-access")})`,
].join("\n");
}
if (error.needsReauthentication) {
const code = error.declineCode ?? "authentication_required";
return [
"### ❌ Pullfrog billing error — card requires 3DS on every charge",
`**Your card issuer requires 3D Secure on every charge** (\`${code}\`).`,
"",
`Your card issuer requires a 3D Secure challenge on each off-session charge (\`${error.declineCode ?? "authentication_required"}\`), which we can't run from the agent. Top up your Router credit balance manually — 3DS runs interactively in Stripe Checkout, and subsequent runs draw from the prepaid balance without triggering another off-session charge.`,
"Pullfrog can't complete a 3DS challenge from inside a workflow. Top up your Router balance once in Stripe Checkout subsequent runs draw from the prepaid balance without re-triggering 3DS.",
"",
"[Top up your Router credit balance →](https://pullfrog.com/console)",
`[Top up balance →](${billingConsoleUrl(owner, "billing")})`,
].join("\n");
}
const codeSuffix = error.declineCode ? ` (\`${error.declineCode}\`)` : "";
return `### ❌ Pullfrog billing error\n\n${error.message}${codeSuffix}\n\n[Manage billing →](https://pullfrog.com/console)`;
if (error.declineCode) {
return [
`**Your card was declined** (\`${error.declineCode}\`).`,
"",
"Update your payment method and Pullfrog will retry on the next run.",
"",
`[Update payment method →](${billingConsoleUrl(owner, "billing")})`,
].join("\n");
}
return [
"**Your Pullfrog balance is empty.**",
"",
"Top up your balance or enable auto-reload to keep runs flowing.",
"",
`[Manage billing →](${billingConsoleUrl(owner, "billing")})`,
].join("\n");
}
/**
* Render a TransientError as user-facing markdown. Distinct framing from
* BillingError so the user doesn't read "❌" and assume their card failed.
* BillingError so the user doesn't read an alarm and assume their card
* failed — this branch is "our fault, retry shortly", not theirs.
*/
function formatTransientErrorSummary(error: TransientError): string {
function formatTransientErrorSummary(error: TransientError, owner: string): string {
return [
"### ⚠️ Pullfrog temporarily unavailable",
"**Pullfrog billing is temporarily unavailable.**",
"",
error.message,
"",
"This is typically transient — the next dispatch should succeed. If it persists, check [status.pullfrog.com](https://status.pullfrog.com).",
`Usually transient — the next dispatch should succeed. If it persists, check [status.pullfrog.com](https://status.pullfrog.com) or [your console](${billingConsoleUrl(owner, "billing")}).`,
].join("\n");
}
@@ -297,6 +364,67 @@ async function resolveProxyModel(ctx: {
log.info(`» proxy: ${label}${ctx.proxyModel}`);
}
/**
* Fetch the most recent persisted PR summary snapshot for this PR.
* Returns null on first-time PRs, when summary is disabled, or on any error.
* Best-effort: a transient API failure should not block the run.
*/
async function fetchPreviousSnapshot(ctx: ToolContext, prNumber: number): Promise<string | null> {
if (!ctx.githubInstallationToken) return null;
try {
const response = await apiFetch({
path: `/api/repo/${ctx.repo.owner}/${ctx.repo.name}/pr/${prNumber}/summary-comment`,
method: "GET",
headers: { authorization: `Bearer ${ctx.githubInstallationToken}` },
signal: AbortSignal.timeout(10_000),
});
if (!response.ok) return null;
const data = (await response.json()) as { snapshot?: string | null };
return typeof data.snapshot === "string" && data.snapshot.length > 0 ? data.snapshot : null;
} catch {
return null;
}
}
/**
* Read the agent-edited PR summary tmpfile and persist to `WorkflowRun.summarySnapshot`.
*
* Best-effort: any failure is logged and does not affect the run's success
* status. Skips the PATCH when the file is byte-identical to its seed —
* persisting the seed verbatim would either re-write what the DB already has
* (on incremental runs) or serialize the placeholder scaffold (on first
* runs), neither of which is useful.
*/
async function persistSummary(ctx: ToolContext): Promise<void> {
const filePath = ctx.toolState.summaryFilePath;
if (!filePath) return;
// already-completed guard: the error-path call (success path persisted,
// then a late step threw) and the SIGINT/SIGTERM handler all funnel
// through here; the first one to arrive wins.
if (ctx.toolState.summaryPersistAttempted) return;
ctx.toolState.summaryPersistAttempted = true;
const snapshot = await readSummaryFile(filePath);
if (!snapshot) {
log.debug(`pr summary tmpfile missing or invalid at ${filePath} — skipping persist`);
return;
}
// soft gate: agent never touched the seeded file. saving the seed back
// is a no-op at best (incremental run — DB already has it) and a bug at
// worst (first run — serializes the placeholder italics). log a warning
// so the failure mode is visible in CI without flipping the run to
// failed.
const seed = ctx.toolState.summarySeed?.trim();
if (seed !== undefined && snapshot === seed) {
log.warning(
"» pr summary tmpfile unchanged from seed — skipping persist (agent did not edit it)"
);
return;
}
await patchWorkflowRunFields(ctx, { summarySnapshot: snapshot }).catch((err) => {
log.debug(`pr summary persist failed: ${err instanceof Error ? err.message : String(err)}`);
});
}
async function writeJobSummary(toolState: ToolState): Promise<void> {
const usageSummary = formatUsageSummary(toolState.usageEntries);
const summaryParts = [toolState.lastProgressBody, usageSummary].filter(Boolean);
@@ -395,7 +523,7 @@ export async function main(): Promise<MainResult> {
});
} catch (error) {
if (error instanceof BillingError) {
const summary = formatBillingErrorSummary(error);
const summary = formatBillingErrorSummary(error, runContext.repo.owner);
await writeSummary(summary).catch(() => {});
// Mirror to the PR progress comment if the trigger created one
// (mention / PR event). Without this, auto-reload declines are only
@@ -405,7 +533,7 @@ export async function main(): Promise<MainResult> {
throw error;
}
if (error instanceof TransientError) {
const summary = formatTransientErrorSummary(error);
const summary = formatTransientErrorSummary(error, runContext.repo.owner);
await writeSummary(summary).catch(() => {});
await reportErrorToComment({ toolState, error: summary }).catch(() => {});
throw error;
@@ -450,6 +578,12 @@ export async function main(): Promise<MainResult> {
const resolvedModel = payload.proxyModel ? undefined : resolveModel({ slug: payload.model });
const agent = resolveAgent({ model: resolvedModel });
// surface the effective model in comment/review footers. payload.model is
// just the stored slug (often undefined for router/oss runs that derive
// the target from proxyModel). matching priority with resolveModelForLog
// so the "Using `…`" badge reflects what actually ran.
toolState.model = payload.proxyModel ?? resolvedModel ?? payload.model;
validateAgentApiKey({
agent,
model: payload.proxyModel ?? resolvedModel ?? payload.model,
@@ -513,6 +647,40 @@ export async function main(): Promise<MainResult> {
log.info(`» MCP server started at ${mcpHttpServer.url}`);
timer.checkpoint("mcpServer");
// seed the rolling PR summary tmpfile when the dispatcher requested it.
// gated on event being a PR — issue/workflow_dispatch runs have no
// summarySnapshot to maintain. file path is exposed to the agent via
// the select_mode response addendum (action/mcp/selectMode.ts).
if (payload.generateSummary && payload.event.is_pr && payload.event.issue_number) {
const previousSnapshot = await fetchPreviousSnapshot(toolContext, payload.event.issue_number);
const filePath = await seedSummaryFile({ tmpdir, previousSnapshot });
toolState.summaryFilePath = filePath;
// capture the exact bytes the agent will see at startup. used by
// the post-run retry loop to detect the agent forgetting to edit
// the file (byte-identical to seed → nudge once via resume turn)
// and by persistSummary to skip the DB write when nothing changed.
// we just wrote the file, so the read shouldn't fail; the catch
// leaves summarySeed unset (its default), in which case the unchanged
// checks downstream are simply skipped.
try {
toolState.summarySeed = await readFile(filePath, "utf8");
} catch {
// intentionally empty — summarySeed stays undefined
}
log.info(
`» summary snapshot seeded at ${filePath} (previous=${previousSnapshot ? "yes" : "no"})`
);
// on SIGINT/SIGTERM we still want to persist whatever the agent has
// written so far. handler is best-effort: any failure inside is
// swallowed by Promise.allSettled in exitHandler.ts, and the
// summaryPersistAttempted guard prevents double-execution if the
// signal arrives after the normal path already persisted. capture a
// narrowed reference so the closure doesn't depend on the outer
// `toolContext` variable being defined later.
const ctxForExit = toolContext;
onExitSignal(() => persistSummary(ctxForExit));
}
startInstallation(toolContext);
const modelForLog = resolveModelForLog({ payload, resolvedModel });
@@ -578,10 +746,11 @@ export async function main(): Promise<MainResult> {
toolState.todoTracker = todoTracker;
// on cancellation, stop scheduling new tracker writes immediately. without this, a
// debounced write queued just before SIGTERM could land at GitHub *after* the post-step
// writes its "This run was cancelled" message, clobbering it back to the task list.
// we can't await in-flight writes (the process is exiting), but cancelling the timer
// shrinks the race window. post-cleanup has its own verify-retry loop for the rest.
// debounced write queued just before SIGTERM could land at GitHub *after* the
// workflow_run.completed webhook has already replaced the comment with the
// "This run was cancelled" body, clobbering it back to the task list. we can't
// await in-flight writes (the process is exiting), but cancelling the timer
// shrinks the race window.
onExitSignal(() => {
todoTracker?.cancel();
});
@@ -624,6 +793,8 @@ export async function main(): Promise<MainResult> {
instructions,
todoTracker,
stopScript: runContext.repoSettings.stopScript,
summaryFilePath: toolState.summaryFilePath,
summarySeed: toolState.summarySeed,
onActivityTimeout: onInnerActivityTimeout,
onToolUse: (event) => {
const wasTracked = recordDiffReadFromToolUse({
@@ -705,20 +876,27 @@ export async function main(): Promise<MainResult> {
});
}
// clean up stranded progress comments. two cases:
// 1. wasUpdated=false: nothing wrote to the comment ("Leaping into action" orphan)
// 2. tracker published a checklist but the agent never wrote a final summary
// (hasPublished=true, finalSummaryWritten=false).
// in both cases, delete the comment so it doesn't linger with stale content.
// wasUpdated is intentionally NOT set here — cleanup is not a real progress update.
// uses finalSummaryWritten (not todoTracker.enabled) so cleanup survives API failures
// in report_progress where cancel() ran but the write didn't succeed.
const trackerWasLastWriter = todoTracker?.hasPublished && !toolState.finalSummaryWritten;
if (
toolContext &&
toolState.progressComment &&
(!toolState.wasUpdated || trackerWasLastWriter)
) {
// read the agent-edited summary tmpfile and persist to the DB. happens
// after the agent exits so the file is in its final state.
if (toolContext) {
await persistSummary(toolContext);
}
// clean up stranded progress comments. the comment is stale unless
// report_progress wrote a final summary to it — three sub-cases all reduce
// to !finalSummaryWritten:
// 1. nothing wrote to the comment ("Leaping into action" orphan)
// 2. tracker published a checklist but the agent never finalized it
// 3. the agent produced a substantive artifact via another MCP write tool
// (create_issue_comment, update_pull_request_body, reply_to_review_comment)
// and skipped report_progress — wasUpdated is true, but the progress
// comment itself was never touched.
// create_pull_request_review owns its own deletion (see action/mcp/review.ts),
// so progressComment is already null by the time we get here for that path.
// uses finalSummaryWritten (not todoTracker.enabled or wasUpdated) so cleanup
// survives API failures in report_progress where cancel() ran but the write
// didn't succeed, and isn't fooled by writes to *other* artifacts.
if (toolContext && toolState.progressComment && !toolState.finalSummaryWritten) {
await deleteProgressComment(toolContext).catch((error) => {
log.debug(`stranded progress comment cleanup failed: ${error}`);
});
@@ -744,16 +922,32 @@ export async function main(): Promise<MainResult> {
killTrackedChildren();
log.error(errorMessage);
// Reclassify OpenRouter "key budget exhausted" mid-run errors as
// BillingError. The agent runtime surfaces this as a generic APIError,
// but it's a Pullfrog billing concern — the user's Router wallet ran
// out partway through the run. Route through the same formatBillingErrorSummary
// path as proxy-token 402s so the user gets actionable copy + a top-up
// CTA on both the job summary and the PR progress comment, instead of
// a generic "❌ Pullfrog failed" stack-trace dump.
const billingError = isRouterKeylimitExhaustedError(errorMessage)
? new BillingError(errorMessage, { code: "router_keylimit_exhausted" })
: null;
// best-effort summary — write the error so it's visible in the Actions summary tab
try {
const errorSummary = `### ❌ Pullfrog failed\n\n\`\`\`\n${errorMessage}\n\`\`\``;
const errorSummary = billingError
? formatBillingErrorSummary(billingError, runContext.repo.owner)
: `### ❌ Pullfrog failed\n\n\`\`\`\n${errorMessage}\n\`\`\``;
const usageSummary = formatUsageSummary(toolState.usageEntries);
const parts = [errorSummary, toolState.lastProgressBody, usageSummary].filter(Boolean);
await writeSummary(parts.join("\n\n"));
} catch {}
try {
await reportErrorToComment({ toolState, error: errorMessage });
const commentBody = billingError
? formatBillingErrorSummary(billingError, runContext.repo.owner)
: errorMessage;
await reportErrorToComment({ toolState, error: commentBody });
} catch {
// error reporting failed, but don't let it mask the original error
}
@@ -765,6 +959,13 @@ export async function main(): Promise<MainResult> {
});
}
// best-effort summary persist on the error path: if the agent successfully
// edited the summary file before timing out / crashing, those edits are
// worth keeping for the next incremental run.
if (toolContext) {
await persistSummary(toolContext);
}
return {
success: false,
error: errorMessage,
+41 -1
View File
@@ -1,5 +1,5 @@
import { createHash } from "node:crypto";
import { writeFileSync } from "node:fs";
import { statSync, unlinkSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import type { Octokit, RestEndpointMethodTypes } from "@octokit/rest";
import { type } from "arktype";
@@ -275,6 +275,39 @@ type CheckoutPrBranchParams = GitContext & {
beforeSha?: string | undefined;
};
// stale lock files left over from a crashed/cancelled prior git process block
// every subsequent fetch with `Unable to create '<path>': File exists`. only
// sweep locks older than this threshold so we never race a concurrent
// legitimate git op that's holding the lock.
const STALE_LOCK_AGE_MS = 30_000;
const GIT_LOCK_PATHS = [
".git/shallow.lock",
".git/index.lock",
".git/objects/maintenance.lock",
] as const;
function cleanupStaleGitLocks(): void {
const now = Date.now();
for (const relPath of GIT_LOCK_PATHS) {
let mtimeMs: number;
try {
mtimeMs = statSync(relPath).mtimeMs;
} catch {
continue;
}
if (now - mtimeMs < STALE_LOCK_AGE_MS) continue;
try {
unlinkSync(relPath);
log.warning(`» removed stale ${relPath} from prior run`);
} catch (e) {
log.debug(
`» failed to remove stale ${relPath}: ${e instanceof Error ? e.message : String(e)}`
);
}
}
}
/**
* Shared helper to checkout a PR branch and configure fork remotes.
* Assumes origin remote is already configured with authentication.
@@ -296,6 +329,12 @@ export async function checkoutPrBranch(
rejectIfLeadingDash(pr.baseRef, "PR base ref");
rejectIfLeadingDash(pr.headRef, "PR head ref");
// self-hosted runners and cancelled jobs frequently leave stale .git/*.lock
// files behind. without this sweep, the first fetch below aborts with
// `Unable to create '.git/shallow.lock': File exists` and the agent has to
// shell out to `rm -f` (issue #564).
cleanupStaleGitLocks();
const isFork = pr.headRepoFullName !== pr.baseRepoFullName;
// always use pr-{number} as local branch name for consistency
@@ -545,6 +584,7 @@ export function CheckoutPrTool(ctx: ToolContext) {
diffPath,
totalLines: countLines({ content: formatResult.content }),
toc: formatResult.toc,
previous: ctx.toolState.diffCoverage,
});
log.debug(
`» diff coverage initialized: diffPath=${diffPath}, totalLines=${ctx.toolState.diffCoverage.totalLines}, tocEntries=${ctx.toolState.diffCoverage.tocEntries.length}`
+17 -51
View File
@@ -12,18 +12,11 @@ import {
import type { ToolContext } from "./server.ts";
import { execute, tool } from "./shared.ts";
/**
* The prefix text for the initial "leaping into action" comment.
* This is used to identify if a comment is still in its initial state
* and hasn't been updated with progress or error messages.
*/
export const LEAPING_INTO_ACTION_PREFIX = "Leaping into action";
export function isLeapingIntoActionCommentBody(body: string): boolean {
const content = stripExistingFooter(body).trimStart();
const firstLine = content.split(/\r?\n/, 1)[0]?.trimEnd() ?? "";
return new RegExp(`(^|\\s)${LEAPING_INTO_ACTION_PREFIX}(\\.\\.\\.)?$`).test(firstLine);
}
// re-export for backward compat with anything importing the leaping helpers from mcp/comment
export {
isLeapingIntoActionCommentBody,
LEAPING_INTO_ACTION_PREFIX,
} from "../utils/leapingComment.ts";
function buildCommentFooter(ctx: ToolContext, customParts?: string[]): string {
const runId = ctx.runId;
@@ -63,10 +56,8 @@ export const Comment = type({
issueNumber: type.number.describe("the issue number to comment on"),
body: type.string.describe("the comment body content"),
type: type
.enumerated("Plan", "Summary", "Comment")
.describe(
"Plan: record as the plan for this run. Summary: record as the PR summary comment (one per PR, updated in place). Comment: regular comment (default)."
)
.enumerated("Plan", "Comment")
.describe("Plan: record as the plan for this run. Comment: regular comment (default).")
.optional(),
});
@@ -74,35 +65,11 @@ export function CreateCommentTool(ctx: ToolContext) {
return tool({
name: "create_issue_comment",
description:
"Create a comment on a GitHub issue or PR. For progress/plan updates on the current run use report_progress instead. Use type: 'Plan' for plan comments, type: 'Summary' for PR summary comments.",
"Create a comment on a GitHub issue or PR. For progress/plan updates on the current run use report_progress instead. Use type: 'Plan' for plan comments.",
parameters: Comment,
execute: execute(async ({ issueNumber, body, type: commentType }) => {
const bodyWithFooter = addFooter(ctx, body);
// if a summary comment already exists (found by select_mode), update instead of creating
if (commentType === "Summary" && ctx.toolState.existingSummaryCommentId) {
log.info(
`» redirecting create_issue_comment(Summary) to update existing comment ${ctx.toolState.existingSummaryCommentId}`
);
const result = await ctx.octokit.rest.issues.updateComment({
owner: ctx.repo.owner,
repo: ctx.repo.name,
comment_id: ctx.toolState.existingSummaryCommentId,
body: bodyWithFooter,
});
if (result.data.node_id) {
await patchWorkflowRunFields(ctx, { summaryCommentNodeId: result.data.node_id });
}
return {
success: true,
commentId: result.data.id,
url: result.data.html_url,
body: result.data.body,
};
}
const result = await ctx.octokit.rest.issues.createComment({
owner: ctx.repo.owner,
repo: ctx.repo.name,
@@ -110,6 +77,8 @@ export function CreateCommentTool(ctx: ToolContext) {
body: bodyWithFooter,
});
ctx.toolState.wasUpdated = true;
if (commentType === "Plan") {
if (result.data.node_id) {
await patchWorkflowRunFields(ctx, { planCommentNodeId: result.data.node_id });
@@ -134,10 +103,6 @@ export function CreateCommentTool(ctx: ToolContext) {
};
}
if (commentType === "Summary" && result.data.node_id) {
await patchWorkflowRunFields(ctx, { summaryCommentNodeId: result.data.node_id });
}
return {
success: true,
commentId: result.data.id,
@@ -212,7 +177,7 @@ export async function reportProgress(
// always track the body for job summary
ctx.toolState.lastProgressBody = body;
// silent events (e.g., auto-label, PR summary) should never create or update progress comments.
// silent events (e.g., auto-label, pr-summary Task) should never create or update progress comments.
// the body is still tracked above for the GitHub Actions job summary.
if (ctx.payload.event.silent) {
return { body, action: "skipped" };
@@ -366,10 +331,6 @@ export function ReportProgressTool(ctx: ToolContext) {
}
const result = await reportProgress(ctx, reportParams);
if (!params.target_plan_comment) {
ctx.toolState.finalSummaryWritten = true;
}
if (result.action === "skipped") {
return {
success: true,
@@ -378,6 +339,10 @@ export function ReportProgressTool(ctx: ToolContext) {
};
}
if (!params.target_plan_comment) {
ctx.toolState.finalSummaryWritten = true;
}
return {
success: true,
...result,
@@ -443,7 +408,8 @@ export function ReplyToReviewCommentTool(ctx: ToolContext) {
body: bodyWithFooter,
});
// mark progress as updated so post script doesn't think the run failed
// mark progress as updated so error reporting + run-result handling know
// a substantive write happened (used by reportErrorToComment / handleAgentResult)
ctx.toolState.wasUpdated = true;
return {
+31 -3
View File
@@ -499,6 +499,21 @@ const GitFetch = type({
depth: type.number.describe("Fetch depth (for shallow clones)").optional(),
});
// when an agent-supplied depth is too shallow to reach the merge base, git
// surfaces "Could not read <sha>" and "remote did not send all necessary
// objects". detect both wordings so a single deepen retry can recover before
// the error reaches the agent (issue #564). git emits the full OID via
// oid_to_hex, so the bound is 40 (SHA-1) or 64 (SHA-256).
const SHALLOW_UNREACHABLE_PATTERNS: RegExp[] = [
/Could not read [a-f0-9]{40,64}/,
/remote did not send all necessary objects/,
];
// large enough to clear the merge base on most real-world PRs without
// downloading the full history; matches the fallback used by checkoutPrBranch
// when the compare API is unavailable.
const DEEPEN_RETRY_DEPTH = 1000;
export function GitFetchTool(ctx: ToolContext) {
return tool({
name: "git_fetch",
@@ -510,9 +525,22 @@ export function GitFetchTool(ctx: ToolContext) {
if (params.depth !== undefined) {
fetchArgs.push(`--depth=${params.depth}`);
}
await $git("fetch", fetchArgs, {
token: ctx.gitToken,
});
try {
await $git("fetch", fetchArgs, { token: ctx.gitToken });
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
const isShallowUnreachable = SHALLOW_UNREACHABLE_PATTERNS.some((p) => p.test(msg));
const isShallow =
isShallowUnreachable &&
$("git", ["rev-parse", "--is-shallow-repository"], { log: false }).trim() === "true";
if (!isShallow) throw err;
log.info(
`» git_fetch hit shallow-unreachable error, retrying with --deepen=${DEEPEN_RETRY_DEPTH}`
);
await $git("fetch", [`--deepen=${DEEPEN_RETRY_DEPTH}`, "--no-tags", "origin", params.ref], {
token: ctx.gitToken,
});
}
return { success: true, ref: params.ref };
}),
});
+2
View File
@@ -49,6 +49,8 @@ export function UpdatePullRequestBodyTool(ctx: ToolContext) {
body: bodyWithFooter,
});
ctx.toolState.wasUpdated = true;
return {
success: true,
number: result.data.number,
+2 -2
View File
@@ -648,8 +648,8 @@ describe("reviewSkipDecision", () => {
describe("duplicateReviewDecision", () => {
// regression: colinhacks/zod#5897 had two reviews submitted from the same
// workflow run 8 seconds apart — a substantive review followed by an empty
// "Reviewed — no issues found." follow-up. the agent re-classified the
// first review's non-blocking observations as "no actionable issues" and
// "No new issues found." follow-up. the agent re-classified the first
// review's non-blocking observations as "no actionable issues" and
// submitted the canonical body per modes.ts. this guard makes the second
// call a no-op without burning a GitHub API call or polluting the PR.
+69 -9
View File
@@ -11,6 +11,7 @@ import {
} from "../utils/diffCoverage.ts";
import { fixDoubleEscapedString } from "../utils/fixDoubleEscapedString.ts";
import { patchWorkflowRunFields } from "../utils/patchWorkflowRunFields.ts";
import { retry } from "../utils/retry.ts";
import { deleteProgressComment } from "./comment.ts";
import type { ToolContext } from "./server.ts";
import { execute, tool } from "./shared.ts";
@@ -21,6 +22,29 @@ function getHttpStatus(err: unknown): number | undefined {
return typeof status === "number" ? status : undefined;
}
/**
* detect GitHub's generic server-side 422 ("An internal error occurred,
* please try again.") that sometimes fires on `POST /pulls/{n}/reviews`.
*
* the body is stable across occurrences and distinct from every other 422
* cause we care about (anchor validation, body length, malformed suggestion
* blocks) — those all cite the specific problem. treating this as a
* transient server error unlocks bounded in-tool retry instead of surfacing
* it to the agent with the generic "likely causes (1)(2)(3)" prompt, which
* induces whack-a-mole comment dropping on content that was never the issue.
*/
export function isTransientReviewError(err: unknown): boolean {
if (getHttpStatus(err) !== 422) return false;
const msg = err instanceof Error ? err.message : String(err);
return /internal error occurred, please try again/i.test(msg);
}
// backoff schedule for transient GitHub 422 "internal error" responses on the
// reviews endpoint. 3 attempts total (initial + 2 retries) with 1s/3s delays
// — most transient GH errors clear within a few seconds, and longer delays
// push review submission past agent-perceived responsiveness.
export const TRANSIENT_REVIEW_RETRY_DELAYS_MS = [1_000, 3_000];
type PullFile = RestEndpointMethodTypes["pulls"]["listFiles"]["response"]["data"][number];
export type CommentableLines = { RIGHT: Set<number>; LEFT: Set<number> };
@@ -187,8 +211,8 @@ export type DuplicateReviewDecision = {
* the agent is instructed to call create_pull_request_review exactly once per
* Review-mode session (see action/modes.ts), but in practice it sometimes
* submits twice — once with substantive feedback, then again with the
* canonical "Reviewed — no issues found." body when the prompt's branch
* logic re-classifies non-blocking observations. the second submission is
* canonical "No new issues found." body when the prompt's branch logic
* re-classifies non-blocking observations. the second submission is
* always redundant: the first review is the record, and the duplicate just
* adds noise to the PR.
*
@@ -483,16 +507,50 @@ export function CreatePullRequestReviewTool(ctx: ToolContext) {
// no body → single-step createReview (no footer needed)
// has body → pending + submit so we can build footer with Fix links using review ID
//
// wrap the submission in `retry` so GitHub's transient 422 "internal
// error" body (distinct from anchor / body-length / suggestion 422s,
// which all cite the specific cause) clears on its own instead of
// surfacing through the generic 422 handler — that framing sent the
// agent dropping valid inline comments chasing a non-issue.
// `shouldRetry` scopes retries to the transient body only, so real
// validation 422s still fail fast.
let result;
try {
result = body
? await createAndSubmitWithFooter(ctx, params, {
body,
approved: approved ?? false,
hasComments: (params.comments?.length ?? 0) > 0,
})
: await createReviewWithStrandedRecovery(ctx, params);
result = await retry(
() =>
body
? createAndSubmitWithFooter(ctx, params, {
body,
approved: approved ?? false,
hasComments: (params.comments?.length ?? 0) > 0,
})
: createReviewWithStrandedRecovery(ctx, params),
{
delaysMs: TRANSIENT_REVIEW_RETRY_DELAYS_MS,
shouldRetry: isTransientReviewError,
label: "review submission",
}
);
} catch (err: unknown) {
// GitHub's transient 422 "internal error" is distinct from anchor /
// body-length / suggestion validation failures — framing it with the
// generic "likely causes (1)(2)(3)" prompt sends the agent dropping
// comments that were never the problem. after bounded in-tool retry
// we surface a dedicated message that tells the agent to wait-and-
// retry or fall back to a body-only review.
if (isTransientReviewError(err)) {
const rawMsg = err instanceof Error ? err.message : String(err);
throw new Error(
`GitHub returned a transient 422 "internal error" on the reviews endpoint after ${TRANSIENT_REVIEW_RETRY_DELAYS_MS.length + 1} attempts. ` +
`This is a GitHub-side issue, not a problem with your review content. ` +
`Do NOT modify or drop inline comments — their content is not the cause. ` +
`Wait ~30 seconds and call this tool once more with the SAME arguments. ` +
`If it still fails, submit a body-only review (move all inline feedback into \`body\` as text) so nothing is lost. ` +
`GitHub said: ${rawMsg}`,
{ cause: err }
);
}
if (getHttpStatus(err) !== 422 || !params.comments?.length) throw err;
const details = params.comments.map((c) => {
@@ -537,6 +595,8 @@ export function CreatePullRequestReviewTool(ctx: ToolContext) {
reviewedSha: actuallyReviewedSha,
};
ctx.toolState.wasUpdated = true;
// a submitted review obsoletes the progress comment — the review IS the
// durable artifact. owned here (not in main.ts) so cleanup is atomic with
// submission and survives any path out of the run (success, timeout,
+39 -61
View File
@@ -1,14 +1,13 @@
import { type } from "arktype";
import { formatMcpToolRef } from "../external.ts";
import { type Mode, PR_SUMMARY_FORMAT } from "../modes.ts";
import type { Mode } from "../modes.ts";
import { apiFetch } from "../utils/apiFetch.ts";
import { log } from "../utils/log.ts";
import type { ToolContext } from "./server.ts";
import { execute, tool } from "./shared.ts";
export const SelectModeParams = type({
mode: type.string.describe(
"the name of the mode to select (e.g., 'Build', 'Plan', 'Review', 'IncrementalReview', 'Fix', 'AddressReviews', 'Task', 'ResolveConflicts', 'Summarize')"
"the name of the mode to select (e.g., 'Build', 'Plan', 'Review', 'IncrementalReview', 'Fix', 'AddressReviews', 'Task', 'ResolveConflicts')"
),
"issue_number?": type("number").describe(
"optional issue number; when provided with Plan mode, used to look up an existing plan comment for this issue (edit vs create)"
@@ -32,18 +31,6 @@ An existing plan comment was found for this issue. Update that comment with the
- produce a structured plan with clear milestones
3. Call \`${t("report_progress")}\` with the full revised plan text and \`{ target_plan_comment: true }\` so it updates the existing plan comment (not the progress comment).
4. Then post a short note to the progress comment (e.g. "Plan has been updated in the comment above.") via \`${t("report_progress")}\` so it is not left as "Leaping...".`,
SummaryUpdate: `### Checklist (updating existing summary)
An existing summary comment was found for this PR. Update it rather than creating a new one.
1. Use \`previousSummaryBody\` from this response as the current summary to revise.
2. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata and a \`diffPath\`.
3. Read the diff using the TOC to selectively read relevant sections. Produce an updated summary reflecting the current state of the PR, using the existing summary (\`previousSummaryBody\`) as a starting point. If EVENT INSTRUCTIONS specify a custom format, follow that instead of the default format below.
4. Call \`${t("edit_issue_comment")}\` with \`commentId: existingSummaryCommentId\` (from this response) and the updated summary body.
5. Call \`${t("report_progress")}\` with a brief note (e.g., "Updated PR summary.").
${PR_SUMMARY_FORMAT}`,
};
}
@@ -78,10 +65,7 @@ function buildOrchestratorGuidance(
// matches the API response for /repo/[owner]/[repo]/issue/[issueNumber]/plan-comment
export type PlanCommentResponsePayload = { error: string } | { commentId: number; body: string };
// matches the API response for /repo/[owner]/[repo]/pr/[prNumber]/summary-comment
export type SummaryCommentResponsePayload = { error: string } | { commentId: number; body: string };
// IMPORTANT: these routes authenticate via GitHub installation token (getEnrichedRepo),
// IMPORTANT: this route authenticates via GitHub installation token (getEnrichedRepo),
// NOT the Pullfrog API JWT (ctx.apiToken). use ctx.githubInstallationToken here.
// see wiki/api-auth.md for the two auth patterns.
async function fetchExistingPlanComment(
@@ -103,33 +87,30 @@ async function fetchExistingPlanComment(
}
}
async function fetchExistingSummaryComment(
ctx: ToolContext,
prNumber: number
): Promise<Extract<SummaryCommentResponsePayload, { commentId: number }> | null> {
if (!ctx.githubInstallationToken) {
log.warning("fetchExistingSummaryComment: no token, skipping");
return null;
}
const path = `/api/repo/${ctx.repo.owner}/${ctx.repo.name}/pr/${prNumber}/summary-comment`;
try {
const response = await apiFetch({
path,
method: "GET",
headers: { authorization: `Bearer ${ctx.githubInstallationToken}` },
signal: AbortSignal.timeout(10_000),
});
const data = (await response.json()) as SummaryCommentResponsePayload;
if (response.ok && "commentId" in data) {
return data;
}
const errMsg = "error" in data ? data.error : "(no error body)";
log.warning(`fetchExistingSummaryComment: ${response.status} ${path}${errMsg}`);
return null;
} catch (error) {
log.warning("fetchExistingSummaryComment failed:", error);
return null;
}
const SUMMARY_MODES = new Set(["Review", "IncrementalReview", "Task"]);
/** modes that gain the PR summary edit step when toolState.summaryFilePath is set.
*
* NOTE: this snapshot is an internal artifact consumed by future agent runs. it is
* deliberately NOT shaped by user-supplied summary instructions — those would warp
* the durable agent context. user-facing summarization (e.g. the review body's
* "Reviewed changes" section) is governed by review-mode prompts and review
* instructions, separately from this snapshot. */
function buildSummaryAddendum(t: (name: string) => string, ctx: ToolContext): string {
const filePath = ctx.toolState.summaryFilePath;
if (!filePath) return "";
return `### PR summary snapshot — required step
A rolling PR summary lives at \`${filePath}\`. It is your durable cross-run agent context — a functional summary of what this PR does, the subsystems and files it touches, the material behavior of its changes, and any risks or open questions worth carrying forward. It is NOT a chronological log of past review runs; commit-level history can already be reconstructed from \`${t("list_pull_request_reviews")}\`.
How to use it:
- read \`${filePath}\` at the START of the run, alongside the diff. it represents what previous agent runs already understood about this PR — absorb it before picking lenses or crafting subagent dispatch prompts. if it's a fresh seed (file is one or two lines), this is a first review and you'll be filling it in from the diff.
- let the snapshot inform triage and dispatch. when it already tracks a risk, your lens prompts to subagents are stronger when they reference that context (e.g. "the JSDoc explicitly scopes to code points — do not flag grapheme-cluster issues" if the snapshot already documents that contract). when something the snapshot tracks is now resolved by new commits, note that. when new commits introduce something the snapshot doesn't yet describe, that's exactly where your fan-out should focus.
- update the file in place to reflect the PR's CURRENT state. revise stale claims, drop resolved risks, add new behavior or risks. accuracy over breadth — every claim must be grounded in the diff. write for the next agent run, not for a human.
- structure however serves THIS PR. there is no required section template. a refactor might organize by renamed export and call-site impact; a feature by capability; a billing change by money path. a compact note of which commit ranges have been reviewed should always be present so future runs scope correctly, but the rest is your call. when the structure works across runs, keep it stable so range-diffs are clean; when the PR's character changes (e.g. scope expands), reshape.
Do NOT call \`${t("create_issue_comment")}\` for the summary — the server reads this file at end-of-run and persists it. The file edit is mandatory regardless of whether a review is submitted; the snapshot feeds the next run.`;
}
export function SelectModeTool(ctx: ToolContext) {
@@ -180,22 +161,19 @@ export function SelectModeTool(ctx: ToolContext) {
}
}
if (selectedMode.name === "Summarize") {
const prNumber = ctx.payload.event.issue_number;
if (prNumber !== undefined) {
const existing = await fetchExistingSummaryComment(ctx, prNumber);
if (existing !== null) {
ctx.toolState.existingSummaryCommentId = existing.commentId;
return {
...buildOrchestratorGuidance(ctx, selectedMode, overrides.SummaryUpdate),
existingSummaryCommentId: existing.commentId,
previousSummaryBody: existing.body,
};
}
}
}
const summaryAddendum = SUMMARY_MODES.has(selectedMode.name)
? buildSummaryAddendum(t, ctx)
: "";
return buildOrchestratorGuidance(ctx, selectedMode);
const base = buildOrchestratorGuidance(ctx, selectedMode);
if (summaryAddendum.length > 0) {
return {
...base,
orchestratorGuidance: `${base.orchestratorGuidance}\n\n${summaryAddendum}`,
summaryFilePath: ctx.toolState.summaryFilePath,
};
}
return base;
}),
});
}
+15 -2
View File
@@ -128,8 +128,21 @@ export interface ToolState {
// set by select_mode when Plan + issue_number and plan-comment API returns existing plan (for report_progress target_plan_comment)
existingPlanCommentId?: number;
previousPlanBody?: string;
// set by select_mode when Summarize mode and summary-comment API returns existing summary
existingSummaryCommentId?: number;
// absolute path to the PR summary markdown file the agent edits in place.
// seeded by main.ts before the agent starts when payload.generateSummary is set;
// read back at end-of-run to persist to DB.
summaryFilePath?: string;
// exact bytes of the seeded snapshot file at run start. compared against
// the file content at end-of-run to detect "agent never touched it" — in
// that case persistSummary skips the DB write (saving the seed verbatim
// would either re-write what the DB already has, on incremental runs, or
// serialize the placeholder scaffold, on first runs).
summarySeed?: string;
// set to true after persistSummary completes once. prevents the error-path
// call (which exists so a successful agent edit before a crash still gets
// persisted) from redundantly re-running the DB PATCH on the
// success-then-late-throw path.
summaryPersistAttempted?: boolean;
output?: string;
usageEntries: AgentUsage[];
model?: string | undefined;
+32 -43
View File
@@ -10,6 +10,12 @@ export interface Mode {
prompt?: string | undefined;
}
// Default user-facing summary format embedded in Review mode review bodies.
// Deliberately scoped to Review (initial PR review). IncrementalReview keeps
// its own terser bullet-list "Reviewed changes" shape since re-review bodies
// are deltas, not introductions. Distinct from the agent-internal snapshot
// (action/utils/prSummary.ts) which has its own stable scaffold and is never
// shaped by user instructions — see selectMode.ts for the firewall.
export const PR_SUMMARY_FORMAT = `### Default format
Follow this structure exactly:
@@ -112,7 +118,7 @@ export function computeModes(agentId: AgentId): Mode[] {
- Do NOT defect-hunt the diff yourself in parallel with the subagent. Your role is dispatch + evaluation; doing the review yourself reintroduces the implementation bias the subagent is meant to mitigate.
- For diffs that rely on third-party API contracts, SDK semantics, framework directives, or DB engine specifics, instruct the subagent to verify load-bearing claims via web search and quote source URLs rather than trust training data — this is the single most common review-quality failure mode.
Review the findings, address valid points, and discard nitpicks or false positives. The reviewer is fallible — it biases toward *recommending additions* (defensive checks for impossible cases, extra logging, new abstractions used once, comments restating code, tests asserting tautologies, "just-in-case" guards). For each finding, ask: would applying it leave the code more sound, correct, AND elegant? Two-out-of-three is not enough — a fix that improves correctness while degrading elegance still degrades the codebase. Reject bloat-shaped findings without applying them, and after applying the rest re-read your diff and be discerning about what *you just changed*: if any fix turned out to be bloat in context, revert it. The goal is code that is sound and correct *while remaining elegant*; the smallest diff that fixes the real defect almost always wins. Then verify only intended changes are present, no debug artifacts or commented-out code remain, no unrelated files were modified. Commit locally via shell (\`git add . && git commit -m "..."\`).
Review the findings, address valid points, and discard nitpicks or false positives. The reviewer is fallible — it biases toward *recommending additions* (defensive checks for impossible cases, extra logging, new abstractions used once, comments restating code, tests asserting tautologies, "just-in-case" guards). For each finding, ask: would applying it leave the code more sound, correct, AND elegant? Two-out-of-three is usually a signal to look harder for a fix that gets all three before settling for one that trades elegance for correctness. Reject bloat-shaped findings without applying them, and after applying the rest re-read your diff and be discerning about what *you just changed*: if any fix turned out to be bloat in context, revert it. The goal is code that is sound and correct *while remaining elegant*; the smallest diff that fixes the real defect almost always wins. Then verify only intended changes are present, no debug artifacts or commented-out code remain, no unrelated files were modified. Commit locally via shell (\`git add . && git commit -m "..."\`).
5. **finalize**:
- confirm a clean working tree, then push via \`${t("push_branch")}\` (see *SYSTEM* Git rules if this fails — prepush errors are usually the repo's tests/lint, not infra timeouts)
@@ -137,7 +143,7 @@ For simple, well-defined tasks, skip the plan phase and go straight to build.`,
3. For each comment:
- understand the feedback
- evaluate whether applying it would leave the code more **sound, correct, AND elegant**. reviewers are fallible and bias toward *recommending additions* (defensive checks for impossible cases, extra abstractions, comments restating obvious code, tests asserting tautologies, "just-in-case" guards). if a request would add bloat — ceremony without commensurate correctness benefit — push back in your reply rather than mechanically applying it. two-out-of-three is not enough; improving correctness while degrading elegance still degrades the code.
- evaluate whether applying it would leave the code more **sound, correct, AND elegant**. reviewers are fallible and bias toward *recommending additions* (defensive checks for impossible cases, extra abstractions, comments restating obvious code, tests asserting tautologies, "just-in-case" guards). if a request would add bloat — ceremony without commensurate correctness benefit — push back in your reply rather than mechanically applying it. two-out-of-three is usually a signal to look harder for a fix that gets all three before settling.
- if the request stands, make the code change using your native tools; otherwise reply explaining why
- record what was done (or why nothing was done)
@@ -175,7 +181,7 @@ ${learningsStep(t, 6)}`,
2. **triage**: orient yourself on the PR — identify *what kind of thing this is* (domain it touches, seams it crosses, external contracts it depends on, user-facing surfaces it changes). orientation only — defer specific defect-hunting to the subagents; pre-reviewing biases the lenses you pick. use \`${t("get_pull_request")}\` and other read-only GitHub tools for additional context if needed.
if the PR is **genuinely trivial**, skip steps 34 entirely and submit \`Reviewed — no issues found.\` per step 5. there's no value in dispatching even one lens for a typo.
if the PR is **genuinely trivial**, skip steps 34 entirely and submit a \`No new issues found.\` review per step 5. there's no value in dispatching even one lens for a typo.
"Genuinely trivial" (skip):
- single-word doc typo, whitespace/format-only, comment-only across any number of files
@@ -243,26 +249,28 @@ ${learningsStep(t, 6)}`,
note: the first create_pull_request_review submission may error with a one-time diff-coverage nudge listing unread TOC regions. retry the same call to proceed — optionally after reading the listed ranges. the pre-flight will not block again this session.
The review body is structured as: \`[optional alert blockquote]\`\`[PR summary using the default format below]\`. Inline comments are passed via the \`comments\` parameter, not in the body.
- **critical issues** (blocks merge — bugs, security, data loss):
\`approved: false\`. Body begins with a GitHub alert blockquote, e.g.:
\`> [!CAUTION]\\n> This PR introduces a race condition in ...\`
Follow with a brief summary if needed. Include all inline comments.
\`approved: false\`. Body opens with \`> [!CAUTION]\\n> This PR introduces ...\`, followed by the PR summary. Include all inline comments via \`comments\`.
- **recommended changes** (non-critical):
\`approved: false\`. Body begins with a GitHub alert blockquote, e.g.:
\`> [!IMPORTANT]\\n> Consider adding input validation for ...\`
Follow with a brief summary if needed. Include all inline comments.
\`approved: false\`. Body opens with \`> [!IMPORTANT]\\n> Consider ...\`, followed by the PR summary. Include all inline comments via \`comments\`.
- **no actionable issues**:
\`approved: true\`, body: "Reviewed — no issues found."`,
\`approved: true\`. Body opens with \`No new issues found.\` followed by the PR summary.
${PR_SUMMARY_FORMAT}`,
},
// IncrementalReview shares Review's multi-lens orchestrator pattern but
// scopes the target to the incremental diff and adds prior-review-feedback
// tracking. The "issues must be NEW since the last Pullfrog review" filter
// lives at aggregation time (step 5), NOT in the subagent prompt — pushing
// the filter into subagents matches the canonical anneal anti-pattern of
// "list known pre-existing failures — don't flag these" and suppresses
// signal on regressions the new commits amplified. The body-format rules
// (Reviewed changes / Prior review feedback) are unchanged from the prior
// version. Same severity-table omission as Review.
// scopes the target to the incremental diff. The "issues must be NEW
// since the last Pullfrog review" filter lives at aggregation time
// (step 5), NOT in the subagent prompt — pushing the filter into
// subagents matches the canonical anneal anti-pattern of "list known
// pre-existing failures — don't flag these" and suppresses signal on
// regressions the new commits amplified. The review body is just
// "Reviewed changes" — a separate "Prior review feedback" checklist
// would duplicate the rolling PR summary snapshot's record of what
// earlier runs already addressed and add noise to the user-facing
// body. Same severity-table omission as Review.
{
name: "IncrementalReview",
description:
@@ -273,7 +281,7 @@ ${learningsStep(t, 6)}`,
2. **incremental scope**: if \`incrementalDiffPath\` is present, read it to see what changed since the last review. this is a range-diff that isolates the net changes, filtering out base branch noise. if not present, fall back to reviewing the full PR diff and determine what changed since Pullfrog's most recent review.
3. **prior feedback**: fetch previous reviews via \`${t("list_pull_request_reviews")}\`. for the most recent Pullfrog review, call \`${t("get_review_comments")}\` with the review ID to retrieve specific prior line-level feedback. you'll need this in step 6 to track which prior comments were addressed.
3. **prior feedback**: fetch previous reviews via \`${t("list_pull_request_reviews")}\`. for the most recent Pullfrog review, call \`${t("get_review_comments")}\` with the review ID to retrieve specific prior line-level feedback. you'll use this to filter your aggregation in step 5 — anything already flagged in a prior review and not changed by the new commits should not be re-raised. you do NOT need to render this in the review body; the rolling PR summary snapshot is the durable record of what's been addressed.
4. **triage & fan out**: orient on the *incremental* changes — domain, seams, external contracts, user-facing surfaces.
@@ -302,20 +310,14 @@ ${learningsStep(t, 6)}`,
5. **aggregate, draft, self-critique**: merge findings; de-dup overlaps; trace each finding yourself. drop praise, style preferences, speculative/unverified claims, findings about pre-existing code unrelated to the new commits, anything not actionable, and anything that re-states prior review feedback (heuristic: if the finding's root cause lives in lines the *new commits* added or modified, it's in scope; otherwise drop). also drop **bloat-shaped findings** — proposed fixes that would add defensive checks for cases that can't happen, abstractions used once, comments restating obvious code, tests asserting tautologies, or "just-in-case" guards. subagents are fallible and bias toward recommending changes; the bar for an actionable inline comment is sound + correct + elegant. recommending a change that improves only one of the three (or degrades elegance to nominally improve correctness) makes the codebase worse, not better. To compute "lines the new commits added or modified": if \`incrementalDiffPath\` from step 1 is present, use it directly. Otherwise, take the prior Pullfrog review's \`commit_id\` (returned alongside each entry from \`${t("list_pull_request_reviews")}\` in step 3) and run \`git diff <prior-review-sha>..HEAD\` to isolate the lines added since that review. draft inline comments with NEW line numbers from the full PR diff — every comment must be actionable, 2-3 sentences max.
then check: which prior review comments were addressed by the new commits? track the addressed ones for step 6b.
6. **build the review body** — a single "Reviewed changes" section: summarize at the logical-change level, not per-file. each bullet starts with a past-tense verb (e.g. \`- Extracted shared CLI runtime into a single module\`, \`- Renamed package to pullfrog\`). avoid file paths unless they add clarity. if the changes can be described in one sentence, use one sentence — no bullets needed. do NOT include a separate "Prior review feedback" checklist; that's tracked in the rolling PR summary snapshot for the next agent run, and surfacing it in the user-facing body is noise (changes that addressed prior feedback are already covered by the Reviewed-changes bullets). in some cases you may receive a complete diff for the whole pull request instead of an incremental one — when this happens, you will need to determine what changes have happened since Pullfrog's most recent review.
6. **build the review body** — two distinct sections:
a. **Reviewed changes**: summarize at the logical-change level, not per-file. each bullet starts with a past-tense verb (e.g. \`- Extracted shared CLI runtime into a single module\`, \`- Renamed package to pullfrog\`). avoid file paths unless they add clarity. if the changes can be described in one sentence, use one sentence — no bullets needed.
b. **Prior review feedback** (only if any were addressed): list only the prior review comments that WERE addressed by the new commits (\`- [x] safeParse instead of parse — addressed\`). omit unaddressed comments. omit this entire section if nothing was addressed. a change can appear in both sections.
- no headings, no tables, no prose paragraphs in either section — just bullets
- in some cases you may receive a complete diff for the whole pull request instead of an incremental one. when this happens, you will need to determine what changes have happened since Pullfrog's most recent review.
7. Submit — Do NOT call \`report_progress\` or \`create_issue_comment\` — the review is the final record and the progress comment will be cleaned up automatically. the review body always includes the reviewed changes from step 6a. append \`Prior review feedback:\\n\` with the checklist from step 6b only if any prior comments were addressed. Follow these rules:
7. Submit — Do NOT call \`report_progress\` or \`create_issue_comment\` — the review is the final record and the progress comment will be cleaned up automatically. Follow these rules:
- note: the first create_pull_request_review submission may error with a one-time diff-coverage nudge listing unread TOC regions. retry the same call to proceed — optionally after reading the listed ranges. the pre-flight will not block again this session.
- IF NO NEW ISSUES, NON-SUBSTANTIVE CHANGES ONLY (trivial formatting, import reordering, comment tweaks): do NOT submit a review. Do NOT call \`report_progress\`. Exit — the progress comment will be cleaned up automatically.
- ELSE IF NEW CRITICAL ISSUES (blocks merge): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with a GitHub alert blockquote (e.g. \`> [!CAUTION]\\n> This PR introduces ...\`), then the reviewed changes summary and prior feedback (if any).
- ELSE IF NEW RECOMMENDED CHANGES (non-critical): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with \`> [!IMPORTANT]\\n> ...\` alert, then the reviewed changes summary and prior feedback (if any).
- ELSE IF NO NEW ISSUES, SUBSTANTIVE CHANGES (new functionality, behavior changes, or fixes to prior review feedback): call \`${t("create_pull_request_review")}\` to create a PR review. If all previous reviews have been properly addressed and no new issues were discovered, you can set \`approved: true\`. body opens with \`No new issues. Reviewed the following changes:\\n\`, then the reviewed changes summary and prior feedback (if any).`,
- ELSE IF NEW CRITICAL ISSUES (blocks merge): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with a GitHub alert blockquote (e.g. \`> [!CAUTION]\\n> This PR introduces ...\`), then the Reviewed-changes summary.
- ELSE IF NEW RECOMMENDED CHANGES (non-critical): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with \`> [!IMPORTANT]\\n> ...\` alert, then the Reviewed-changes summary.
- ELSE IF NO NEW ISSUES, SUBSTANTIVE CHANGES (new functionality, behavior changes, or fixes to prior review feedback): call \`${t("create_pull_request_review")}\` to create a PR review. If all previous reviews have been properly addressed and no new issues were discovered, you can set \`approved: true\`. body opens with \`No new issues. Reviewed the following changes:\\n\`, then the Reviewed-changes summary.`,
},
{
name: "Plan",
@@ -405,19 +407,6 @@ ${learningsStep(t, 6)}`,
${learningsStep(t, 4)}`,
},
{
name: "Summarize",
description:
"Summarize a PR with a structured comment that is updated in place on subsequent pushes",
prompt: `### Checklist
1. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata and a \`diffPath\`.
2. Read the diff using the TOC to selectively read relevant sections (not the entire file). Produce a structured summary. If EVENT INSTRUCTIONS specify a custom format, follow that instead of the default format below.
3. Call \`${t("create_issue_comment")}\` with \`type: "Summary"\` and the summary body.
4. Call \`${t("report_progress")}\` with a brief note (e.g., "Posted PR summary.").
${PR_SUMMARY_FORMAT}`,
},
];
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "pullfrog",
"version": "0.0.204",
"version": "0.1.1",
"type": "module",
"bin": {
"pullfrog": "dist/cli.mjs",
+1 -8
View File
@@ -12,7 +12,6 @@ import { log } from "./utils/cli.ts";
import { runInDocker } from "./utils/docker.ts";
import { ensureGitHubToken } from "./utils/github.ts";
import { isInsideDocker } from "./utils/globals.ts";
import { runPostCleanup } from "./utils/postCleanup.ts";
import { setupTestRepo } from "./utils/setup.ts";
/**
@@ -78,13 +77,7 @@ export async function run(inputsOrPrompt: Inputs | string): Promise<AgentResult>
}
}
// wrap main() so post cleanup runs even on failure (mirrors action.yml post-if: "failure() || cancelled()")
let result: AgentResult;
try {
result = await main();
} finally {
await runPostCleanup();
}
const result: AgentResult = await main();
process.chdir(originalCwd);
-8
View File
@@ -1,8 +0,0 @@
#!/usr/bin/env node
import { runPullfrogCli } from "./runCli.ts";
runPullfrogCli({
cliArgs: ["gha", "--post"],
swallowErrors: true,
});
-1
View File
@@ -7,7 +7,6 @@ const scriptDir = dirname(fileURLToPath(import.meta.url));
const entryPoints = [
resolve(scriptDir, "../entry.ts"),
resolve(scriptDir, "../post.ts"),
resolve(scriptDir, "../get-installation-token/entry.ts"),
resolve(scriptDir, "../get-installation-token/post.ts"),
];
@@ -11,8 +11,8 @@ exports[`latest model per provider snapshot > matches snapshot 1`] = `
"releaseDate": "2026-04-24",
},
"google": {
"modelId": "gemma-4-31b-it",
"releaseDate": "2026-04-02",
"modelId": "gemini-3.1-flash-lite",
"releaseDate": "2026-05-07",
},
"moonshotai": {
"modelId": "kimi-k2.6",
+8 -2
View File
@@ -1,4 +1,4 @@
import { resolveDisplayAlias } from "../models.ts";
import { modelAliases, resolveDisplayAlias } from "../models.ts";
export const PULLFROG_DIVIDER = "<!-- PULLFROG_DIVIDER_DO_NOT_REMOVE_PLZ -->";
@@ -28,7 +28,13 @@ export interface BuildPullfrogFooterParams {
function formatModelLabel(slug: string): string {
// walk the fallback chain so a deprecated stored slug shows the model the
// run actually executed against (e.g. "GPT", not "GPT Codex").
const alias = resolveDisplayAlias(slug);
const alias =
resolveDisplayAlias(slug) ??
// reverse-lookup: when the caller passes an effective model (proxy or
// resolved target like "openrouter/anthropic/claude-opus-4.7") instead of
// a stored alias slug, find the alias whose resolve target matches so we
// still render a friendly display name.
modelAliases.find((a) => a.resolve === slug || a.openRouterResolve === slug);
if (!alias) return `\`${slug}\``;
return alias.isFree ? `\`${alias.displayName}\` (free)` : `\`${alias.displayName}\``;
}
+18
View File
@@ -98,6 +98,24 @@ describe("diff coverage line checker", () => {
]);
});
it("carries forward coveragePreflightRan from a previous state across checkout refreshes", () => {
const previous = createDiffCoverageState({ diffPath, totalLines: 30, toc });
previous.coveragePreflightRan = true;
previous.coveredRanges = [{ startLine: 5, endLine: 10 }];
const next = createDiffCoverageState({ diffPath, totalLines: 50, toc, previous });
expect(next.coveragePreflightRan).toBe(true);
// coveredRanges are tied to the previous diff content and must not leak forward
expect(next.coveredRanges).toEqual([]);
expect(next.totalLines).toBe(50);
});
it("defaults coveragePreflightRan to false when no previous state is provided", () => {
const state = createDiffCoverageState({ diffPath, totalLines: 30, toc });
expect(state.coveragePreflightRan).toBe(false);
});
it("computes per-file unread ranges from tracked reads", () => {
const state = createDiffCoverageState({
diffPath,
+5 -1
View File
@@ -78,13 +78,17 @@ export function createDiffCoverageState(params: {
diffPath: string;
totalLines: number;
toc: string;
previous?: DiffCoverageState | undefined;
}): DiffCoverageState {
return {
diffPath: params.diffPath,
totalLines: params.totalLines,
tocEntries: parseDiffTocEntries({ toc: params.toc }),
coveredRanges: [],
coveragePreflightRan: false,
// carry forward across checkout_pr refreshes so the nudge stays "once per
// review session". coveredRanges are intentionally not carried because
// line numbers are tied to the previous diff's content.
coveragePreflightRan: params.previous?.coveragePreflightRan ?? false,
};
}
+1 -1
View File
@@ -289,7 +289,7 @@ When embedding images (e.g. uploaded screenshots) in comments or PR bodies, alwa
**\`report_progress\`**: call this exactly once at the end of every run with a brief final summary (1-3 sentences) unless the mode guidance instructs otherwise. Never call it for intermediate status updates (e.g., "Checking for changes...", "Starting review...") — the task list handles live progress automatically. Calling \`report_progress\` replaces the task list with your summary and preserves the current task list in a collapsible section. Keep the summary concise — do not repeat what the task list already shows. Focus on the outcome (what was accomplished, links to artifacts) rather than listing individual steps. If something failed, include the tool's error text even when that makes the summary longer.
Never use \`create_issue_comment\` for task progress — that creates duplicate comments and leaves the progress comment stuck in its initial state. \`create_issue_comment\` is only for standalone comments unrelated to your current task (e.g., Plan comments, PR Summary comments).
Never use \`create_issue_comment\` for task progress — that creates duplicate comments and leaves the progress comment stuck in its initial state. \`create_issue_comment\` is only for standalone comments unrelated to your current task (e.g., Plan comments).
### If you get stuck
+18
View File
@@ -0,0 +1,18 @@
import { stripExistingFooter } from "./buildPullfrogFooter.ts";
/**
* The prefix text for the initial "leaping into action" comment.
* Used to detect whether a progress comment is still in its initial state
* and hasn't been updated with real progress or error messages.
*
* Lives in `utils/` (not `mcp/`) so it can be re-exported via `pullfrog/internal`
* without dragging the MCP server's transitive imports into the Next.js app's
* type-check graph.
*/
export const LEAPING_INTO_ACTION_PREFIX = "Leaping into action";
export function isLeapingIntoActionCommentBody(body: string): boolean {
const content = stripExistingFooter(body).trimStart();
const firstLine = content.split(/\r?\n/, 1)[0]?.trimEnd() ?? "";
return new RegExp(`(^|\\s)${LEAPING_INTO_ACTION_PREFIX}(\\.\\.\\.)?$`).test(firstLine);
}
+2 -2
View File
@@ -14,7 +14,7 @@ export type WorkflowRunArtifactPatchKey =
| "issueNodeId"
| "reviewNodeId"
| "planCommentNodeId"
| "summaryCommentNodeId";
| "summarySnapshot";
/**
* Usage fields aggregated across all agent calls and PATCHed once at
@@ -37,7 +37,7 @@ const STRING_KEYS: WorkflowRunArtifactPatchKey[] = [
"issueNodeId",
"reviewNodeId",
"planCommentNodeId",
"summaryCommentNodeId",
"summarySnapshot",
];
const NUMBER_KEYS: WorkflowRunUsagePatchKey[] = [
+2
View File
@@ -27,6 +27,7 @@ export const JsonPayload = type({
id: "string",
type: "'issue' | 'review'",
}).or("undefined"),
"generateSummary?": "boolean | undefined",
});
// permission levels that indicate collaborator status (have push access)
@@ -160,6 +161,7 @@ export function resolvePayload(
timeout: inputs.timeout ?? jsonPayload?.timeout,
cwd: resolveCwd(inputs.cwd),
progressComment: jsonPayload?.progressComment,
generateSummary: jsonPayload?.generateSummary,
// permissions: inputs > repoSettings > fallbacks
push: inputs.push ?? repoSettings.push ?? "restricted",
-246
View File
@@ -1,246 +0,0 @@
import { isLeapingIntoActionCommentBody } from "../mcp/comment.ts";
import { getApiUrl } from "./apiUrl.ts";
import { buildPullfrogFooter } from "./buildPullfrogFooter.ts";
import { log } from "./cli.ts";
import { createOctokit, parseRepoContext } from "./github.ts";
import { type ResolvedPromptInput, resolvePromptInput } from "./payload.ts";
import {
getProgressComment,
type ProgressComment,
parseProgressComment,
updateProgressComment,
} from "./progressComment.ts";
import { getJobToken } from "./token.ts";
type JsonPromptInput = Extract<ResolvedPromptInput, object>; // not string
interface PostCleanupContext {
repoContext: ReturnType<typeof parseRepoContext>;
octokit: ReturnType<typeof createOctokit>;
runId: number | undefined;
promptInput: JsonPromptInput | null;
}
// controls whether the script should check the reason for the workflow termination.
// it can be either canceled or failed.
// YAML file cannot supply it (not in ENV), so an extra request is required to check it.
const SHOULD_CHECK_REASON = true;
function buildErrorCommentBody(ctx: PostCleanupContext, isCancellation: boolean): string {
let errorMessage = isCancellation
? `This run was cancelled 🛑\n\nThe workflow was cancelled before completion.`
: `This run croaked 😵\n\nThe workflow encountered an error before any progress could be reported.`;
if (ctx.runId) {
errorMessage += " Please check the link below for details.";
}
const customParts: string[] = [];
if (!isCancellation && ctx.runId) {
const apiUrl = getApiUrl();
customParts.push(
`[Rerun failed job ➔](${apiUrl}/trigger/${ctx.repoContext.owner}/${ctx.repoContext.name}/${ctx.runId}?action=rerun)`
);
}
const footer = buildPullfrogFooter({
triggeredBy: true,
workflowRun: ctx.runId
? {
owner: ctx.repoContext.owner,
repo: ctx.repoContext.name,
runId: ctx.runId,
}
: undefined,
customParts,
});
return `${errorMessage}${footer}`;
}
async function validateStuckProgressComment(
ctx: PostCleanupContext
): Promise<ProgressComment | null> {
const promptComment = ctx.promptInput?.progressComment;
if (!promptComment) {
log.info("[post] no progressComment in prompt input, skipping cleanup");
return null;
}
const comment = parseProgressComment(promptComment);
if (!comment) {
log.info(`[post] progressComment.id is not a positive integer: ${promptComment.id}`);
return null;
}
log.info(`[post] validating progressComment from prompt input: ${comment.id} (${comment.type})`);
try {
const fetched = await getProgressComment(
{ octokit: ctx.octokit, owner: ctx.repoContext.owner, repo: ctx.repoContext.name },
comment
);
const body = fetched.body ?? "";
if (isLeapingIntoActionCommentBody(body)) {
log.info(`[post] comment ${comment.id} is stuck on "Leaping into action"`);
return comment;
}
// detect stranded todo checklists left by the tracker when the process was killed
// before the agent could call report_progress with a final summary
if (/^- \[[ x]\] |^- \*\*→\*\* |^- ~~/.test(body)) {
log.info(`[post] comment ${comment.id} is stuck on a todo checklist`);
return comment;
}
log.info(`[post] comment ${comment.id} is not stuck (already updated or different content)`);
return null;
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
log.info(`[post] failed to get comment ${comment.id}: ${errorMessage}`);
return null;
}
}
async function getIsCancelled(ctx: PostCleanupContext): Promise<boolean> {
if (!ctx.runId) return false; // can't check without a run ID — assume failure
try {
const jobsResult = await ctx.octokit.rest.actions.listJobsForWorkflowRun({
owner: ctx.repoContext.owner,
repo: ctx.repoContext.name,
run_id: ctx.runId,
});
// find current job by matching GITHUB_JOB env var.
// GITHUB_JOB is the job ID (yaml key), but job.name is the display name.
// for matrix jobs, the name includes matrix values like "build (ubuntu-latest, node-18)"
// so we match jobs that START with the job ID
const currentJobName = process.env.GITHUB_JOB;
const currentJob = currentJobName
? jobsResult.data.jobs.find(
(j) => j.name === currentJobName || j.name.startsWith(`${currentJobName} (`)
)
: jobsResult.data.jobs[0]; // fallback to first job
if (!currentJob) {
log.warning("[post] could not find current job");
return false;
}
log.info(`[post] job status: ${currentJob.status}, conclusion: ${currentJob.conclusion}`);
if (currentJob.conclusion === "cancelled") return true; // whole job explicit cancellation
// but if it's still null, check steps for cancellation:
const cancelledStep = currentJob.steps?.find((step) => step.conclusion === "cancelled");
if (cancelledStep) {
log.info(`[post] found cancelled step: ${cancelledStep.name}`);
return true;
}
log.info("[post] no cancellation found, assuming failure");
} catch (error) {
log.info(
`[post] failed to get job status: ${error instanceof Error ? error.message : String(error)}`
);
}
return false; // assuming failure
}
export async function runPostCleanup(): Promise<void> {
log.info("» [post] starting post cleanup");
const runId = process.env.GITHUB_RUN_ID
? Number.parseInt(process.env.GITHUB_RUN_ID, 10)
: undefined;
// resolve prompt input once and use it for both issue number and comment ID extraction
// only use the object form (JSON payload), not plain string prompts
let promptInput: JsonPromptInput | null = null;
try {
const resolved = resolvePromptInput();
if (typeof resolved !== "string") promptInput = resolved;
} catch (error) {
log.info(
`[post] failed to resolve prompt input: ${error instanceof Error ? error.message : String(error)}`
);
}
// get job token for API calls
const token = getJobToken();
const repoContext = parseRepoContext();
const octokit = createOctokit(token);
const ctx: PostCleanupContext = { repoContext, octokit, runId, promptInput };
const stuck = await validateStuckProgressComment(ctx);
if (!stuck) return log.info("» [post] no stuck progress comment to update, skipping cleanup");
log.info(
`» [post] validated stuck comment: ${stuck.id} (${stuck.type}), updating with error message`
);
try {
const body = buildErrorCommentBody(
ctx,
SHOULD_CHECK_REASON ? await getIsCancelled(ctx) : false
);
await writeAndVerify(ctx, stuck, body);
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
log.info(`[post] failed to update comment: ${errorMessage}`);
}
}
// post-cleanup runs in a separate process from the cancelled action, so any in-flight
// HTTP write the action's todoTracker had on the wire when SIGTERM landed can still get
// processed by GitHub *after* our update — clobbering the cancellation message back to
// the stale task list. (action-side mitigation: SIGTERM handler cancels the tracker; here
// we close the remaining race by reading back our write and re-issuing if it lost.)
const VERIFY_DELAY_MS = 3000;
const MAX_WRITE_ATTEMPTS = 3;
async function writeAndVerify(
ctx: PostCleanupContext,
comment: ProgressComment,
body: string
): Promise<void> {
const apiCtx = {
octokit: ctx.octokit,
owner: ctx.repoContext.owner,
repo: ctx.repoContext.name,
};
for (let attempt = 1; attempt <= MAX_WRITE_ATTEMPTS; attempt++) {
await updateProgressComment(apiCtx, comment, body);
await new Promise((resolve) => setTimeout(resolve, VERIFY_DELAY_MS));
let fetched: Awaited<ReturnType<typeof getProgressComment>>;
try {
fetched = await getProgressComment(apiCtx, comment);
} catch (error) {
// verify GET failed (5xx, secondary rate limit, network blip). the PUT itself
// returned 200, so we trust it landed; another write-and-verify pass would just
// amplify writes against a flaky GitHub. log and exit — if a stale tracker write
// does clobber us, the comment will be wrong but the agent's commit + replies
// already conveyed the substance of the run.
log.warning(
`[post] verify GET failed after attempt ${attempt} — trusting our PUT landed: ${
error instanceof Error ? error.message : String(error)
}`
);
return;
}
if (fetched.body === body) {
log.info(
`» [post] successfully updated progress comment (attempt ${attempt}/${MAX_WRITE_ATTEMPTS})`
);
return;
}
log.info(
`[post] body was overwritten after our write (attempt ${attempt}/${MAX_WRITE_ATTEMPTS}), retrying`
);
}
log.warning(
`[post] gave up after ${MAX_WRITE_ATTEMPTS} attempts — comment may be stale (in-flight writes from the cancelled run kept clobbering us)`
);
}
+78
View File
@@ -0,0 +1,78 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
/**
* The PR-level summary snapshot is a markdown file the agent edits in place
* during a Review / IncrementalReview run. The server seeds the file with
* either the previous run's snapshot (incremental) or a stub scaffold (first
* run), lets the agent edit it with its native file-editing tools, then
* reads it back at end-of-run and persists it to `WorkflowRun.summarySnapshot`.
*
* The snapshot is an internal artifact it is consumed by future agent runs
* as durable cross-run context, not surfaced to humans. User-visible summary
* content lives in the Review / IncrementalReview review bodies, governed by
* `action/modes.ts`.
*
* Edit-in-place avoids the output-token tax of a tool call that regurgitates
* the full snapshot, and gives incremental runs a clean surface that
* range-diffs cleanly across runs because the section headings are stable.
*/
export const SUMMARY_FILE_NAME = "pullfrog-summary.md";
/**
* minimal seed for first-run PRs. just a header + a one-line note about
* what this file is for. structure is intentionally NOT prescribed
* different PRs warrant different organization, and the agent should pick
* a shape that fits this PR. the agent's prompt (see selectMode.ts
* `buildSummaryAddendum`) carries the actual instructions for what to
* capture and how.
*
* keeping the seed short also makes the unchanged-from-seed gate more
* sensitive any meaningful edit moves the file off the seed, so
* `persistSummary` can reliably skip the DB write when the agent didn't
* touch the file.
*/
export const SUMMARY_SCAFFOLD = `# PR summary
<!-- durable cross-run context. edit in place; the next agent run reads this
before reviewing new commits. structure however serves the PR best. -->
`;
const MIN_SNAPSHOT_LENGTH = 60;
/** PG TEXT can hold ~1GB but a sane cap protects the DB / API payloads. */
const MAX_SNAPSHOT_LENGTH = 32_768;
export function summaryFilePath(tmpdir: string): string {
return join(tmpdir, SUMMARY_FILE_NAME);
}
/** seed the summary file with previous snapshot (incremental) or scaffold (first run). */
export async function seedSummaryFile(params: {
tmpdir: string;
previousSnapshot: string | null;
}): Promise<string> {
const path = summaryFilePath(params.tmpdir);
await mkdir(dirname(path), { recursive: true });
const seed =
params.previousSnapshot && params.previousSnapshot.trim().length >= MIN_SNAPSHOT_LENGTH
? params.previousSnapshot
: SUMMARY_SCAFFOLD;
await writeFile(path, seed, "utf8");
return path;
}
/** read + validate the summary file written by the agent.
* returns null when the file is missing or fails sanity checks. */
export async function readSummaryFile(path: string): Promise<string | null> {
let raw: string;
try {
raw = await readFile(path, "utf8");
} catch {
return null;
}
const trimmed = raw.trim();
if (trimmed.length < MIN_SNAPSHOT_LENGTH) return null;
if (trimmed.length > MAX_SNAPSHOT_LENGTH) return trimmed.slice(0, MAX_SNAPSHOT_LENGTH);
return trimmed;
}
+127
View File
@@ -0,0 +1,127 @@
import { detectProviderError, isRouterKeylimitExhaustedError } from "./providerErrors.ts";
describe("detectProviderError", () => {
describe("false positives previously seen in production", () => {
it("returns null for commit SHAs containing 429", () => {
expect(detectProviderError("hash=7a46d89f505b36df49b4f54429daffa1a9459b11")).toBeNull();
expect(detectProviderError("commit f609cc89e84596ab125d60dac568bfb2ef398396 429")).toBeNull();
});
it("returns null for x-ratelimit-* response headers in 401 error JSON", () => {
const stderr = JSON.stringify({
error: { name: "APIError", statusCode: 401, message: "Invalid authentication credentials" },
headers: {
"x-ratelimit-limit-requests": 50,
"x-ratelimit-remaining-requests": 49,
"x-ratelimit-reset-tokens": "2025-01-01T00:00:00Z",
},
});
expect(detectProviderError(stderr)).toBeNull();
});
it("returns null for INTERNAL_SERVER_ERROR substring", () => {
expect(detectProviderError("HTTP/1.1 500 INTERNAL_SERVER_ERROR")).toBeNull();
expect(detectProviderError("expected: not INTERNAL_SERVER_ERROR")).toBeNull();
});
it("returns null for INTERNALS substring", () => {
expect(detectProviderError("debugging INTERNALS of the parser")).toBeNull();
});
});
describe("real provider errors", () => {
it("detects 429 only when adjacent to a status key", () => {
expect(detectProviderError('{"statusCode": 429}')).toBe("rate limited (429)");
expect(detectProviderError('{"status_code": 429, "message": "..."}')).toBe(
"rate limited (429)"
);
expect(detectProviderError("http_status: 429")).toBe("rate limited (429)");
expect(detectProviderError("status=429")).toBe("rate limited (429)");
});
it("detects rate_limit_error and rate_limit_exceeded", () => {
expect(detectProviderError('{"type":"rate_limit_error"}')).toBe("rate limited");
expect(detectProviderError("rate_limit_exceeded")).toBe("rate limited");
expect(detectProviderError("plain rate limit reached")).toBe("rate limited");
});
it("detects rate-limit phrasing with trailing inflection", () => {
expect(detectProviderError("Error: rate limited by provider")).toBe("rate limited");
expect(detectProviderError("rate limits exceeded for this key")).toBe("rate limited");
});
it("detects RESOURCE_EXHAUSTED", () => {
expect(detectProviderError('"status": "RESOURCE_EXHAUSTED"')).toBe("quota exhausted");
});
it("detects gRPC INTERNAL status as a whole word", () => {
expect(detectProviderError('"status": "INTERNAL"')).toBe("provider internal error");
});
it("detects UNAVAILABLE as a whole word", () => {
expect(detectProviderError('"status": "UNAVAILABLE"')).toBe("provider unavailable");
});
it("detects 500 / 503 only when adjacent to a status key", () => {
expect(detectProviderError('"statusCode": 500')).toBe("provider 500 error");
expect(detectProviderError('"statusCode": 503')).toBe("provider unavailable (503)");
expect(detectProviderError("v1.503.0 release notes")).toBeNull();
});
it("detects quota and zero-quota responses", () => {
expect(detectProviderError('"message": "quota exceeded"')).toBe("quota error");
expect(detectProviderError('{"code":"insufficient_quota"}')).toBe("quota error");
expect(detectProviderError('"error":"quota_exceeded"')).toBe("quota error");
expect(detectProviderError('{"reason":"quotaExceeded"}')).toBe("quota error");
expect(detectProviderError('{"limit": 0, "remaining": 0}')).toBe("zero quota");
expect(detectProviderError('"time_limit": 0')).toBeNull();
});
});
});
describe("isRouterKeylimitExhaustedError", () => {
it("matches the canonical OpenRouter mid-run error", () => {
expect(
isRouterKeylimitExhaustedError(
"APIError: This request requires more credits, or fewer max_tokens. " +
"You requested up to 32000 tokens, but can only afford 22800. " +
"To increase, visit https://openrouter.ai/settings/keys and create a key with a higher total limit"
)
).toBe(true);
});
it("matches the 'requires more credits' phrasing on its own", () => {
expect(
isRouterKeylimitExhaustedError("This request requires more credits, or fewer max_tokens.")
).toBe(true);
});
it("matches the 'requested up to ... can only afford' phrasing on its own", () => {
expect(
isRouterKeylimitExhaustedError("You requested up to 8000 tokens but can only afford 1234")
).toBe(true);
});
it("does not match generic out-of-credit text", () => {
expect(isRouterKeylimitExhaustedError("Your account has insufficient credits")).toBe(false);
expect(isRouterKeylimitExhaustedError("rate_limit_exceeded")).toBe(false);
expect(isRouterKeylimitExhaustedError('{"limit": 0}')).toBe(false);
});
it("does not match unrelated mentions of max_tokens", () => {
expect(isRouterKeylimitExhaustedError("max_tokens parameter must be a positive integer")).toBe(
false
);
});
it("matches across newlines (defends against upstream wrapping/reformatting)", () => {
expect(
isRouterKeylimitExhaustedError(
"APIError: This request requires more credits, or\nfewer max_tokens. You requested up to 32000 tokens"
)
).toBe(true);
expect(
isRouterKeylimitExhaustedError("You requested up to 32000 tokens,\nbut can only afford 22800")
).toBe(true);
});
});
+57 -11
View File
@@ -1,18 +1,64 @@
const PROVIDER_ERROR_PATTERNS = [
{ pattern: "429", label: "rate limited (429)" },
{ pattern: "RESOURCE_EXHAUSTED", label: "quota exhausted" },
{ pattern: "quota", label: "quota error" },
{ pattern: "status: 500", label: "provider 500 error" },
{ pattern: "INTERNAL", label: "provider internal error" },
{ pattern: "status: 503", label: "provider unavailable (503)" },
{ pattern: "UNAVAILABLE", label: "provider unavailable" },
{ pattern: "rate limit", label: "rate limited" },
{ pattern: "limit: 0", label: "zero quota" },
type ProviderErrorPattern = { regex: RegExp; label: string };
// status codes are only treated as provider errors when they are adjacent to
// a recognised status key. this rejects commit SHAs that happen to contain
// "429", version strings, file hashes, etc.
const statusKey = `\\b(?:status[_ ]?code|http[_ ]?status|status)["']?\\s*[:=]\\s*["']?`;
const PROVIDER_ERROR_PATTERNS: ProviderErrorPattern[] = [
{ regex: new RegExp(`${statusKey}429\\b`, "i"), label: "rate limited (429)" },
{ regex: new RegExp(`${statusKey}500\\b`, "i"), label: "provider 500 error" },
{ regex: new RegExp(`${statusKey}503\\b`, "i"), label: "provider unavailable (503)" },
// matches `rate limit`, `rate limited`, `rate limits exceeded`,
// `rate_limit_error`, `rate_limit_exceeded`. the leading `\b` + `[_ ]`
// separator rejects `x-ratelimit-*` / `anthropic-ratelimit-*` response
// headers (no separator between "rate" and "limit") which routinely
// appear in dumped 401 / 4xx error JSON.
{ regex: /\brate[_ ]limit/i, label: "rate limited" },
{ regex: /\bRESOURCE_EXHAUSTED\b/, label: "quota exhausted" },
// Google gRPC `INTERNAL` status. word-boundary anchors reject
// `INTERNAL_SERVER_ERROR` (HTTP 500 message that may appear in unrelated
// log lines) and identifiers like `INTERNALS`.
{ regex: /\bINTERNAL\b/, label: "provider internal error" },
{ regex: /\bUNAVAILABLE\b/, label: "provider unavailable" },
// matches `quota`, `insufficient_quota`, `quota_exceeded`, `quotaExceeded`.
// word-character lookarounds would reject `_quota` / `quotaX`; `quota` is
// specific enough that a plain substring match is safe.
{ regex: /quota/i, label: "quota error" },
// explicit zero-quota response, e.g. `{"limit": 0}`. the `\b` anchor
// around `limit` rejects keys like `time_limit` or `field_limit`.
{ regex: /["']?\blimit\b["']?\s*:\s*0\b/, label: "zero quota" },
];
export function detectProviderError(text: string): string | null {
for (const entry of PROVIDER_ERROR_PATTERNS) {
if (text.includes(entry.pattern)) return entry.label;
if (entry.regex.test(text)) return entry.label;
}
return null;
}
/**
* OpenRouter's response when the per-run key's remaining budget can't cover
* the agent's `max_tokens` reservation. Distinct from a generic provider error
* because it's a Pullfrog billing concern, not an upstream outage — the user's
* Router wallet ran out (or the key budget was undersized at mint time and the
* agent ran out of headroom partway through).
*
* Match must be specific to this exact OpenRouter error class. Generic "credits"
* or "limit" text shows up in unrelated errors and would mis-classify them.
*
* Sample:
* `APIError: This request requires more credits, or fewer max_tokens.
* You requested up to 32000 tokens, but can only afford 22800.`
*/
// `/s` (dotAll) lets `.*?` cross newlines so we still detect the error if any
// upstream layer reformats the message onto multiple lines. Without it, a
// single inserted `\n` would silently bypass the BillingError reclassification
// and the user would see the generic `❌ Pullfrog failed` dump instead of the
// actionable top-up CTA.
const ROUTER_KEYLIMIT_EXHAUSTED_PATTERN =
/requires more credits.*?fewer max_tokens|requested up to \d+ tokens.*?can only afford/is;
export function isRouterKeylimitExhaustedError(text: string): boolean {
return ROUTER_KEYLIMIT_EXHAUSTED_PATTERN.test(text);
}
+14 -3
View File
@@ -4,6 +4,12 @@ import { log } from "./cli.ts";
export type RetryOptions = {
maxAttempts?: number;
delayMs?: number;
/**
* explicit delay schedule one entry per retry (length N N+1 attempts).
* when set, overrides `maxAttempts` and `delayMs`. e.g. `[1_000, 3_000]`
* means up to 3 attempts, sleeping 1s before retry 2 and 3s before retry 3.
*/
delaysMs?: readonly number[];
shouldRetry?: (error: unknown) => boolean;
label?: string;
};
@@ -20,10 +26,15 @@ const defaultShouldRetry = (error: unknown): boolean => {
};
export async function retry<T>(fn: () => Promise<T>, options: RetryOptions = {}): Promise<T> {
const maxAttempts = options.maxAttempts ?? 3;
const delayMs = options.delayMs ?? 1000;
const shouldRetry = options.shouldRetry ?? defaultShouldRetry;
const label = options.label ?? "operation";
const delays = options.delaysMs
? Array.from(options.delaysMs)
: Array.from(
{ length: (options.maxAttempts ?? 3) - 1 },
(_, i) => (options.delayMs ?? 1000) * (i + 1)
);
const maxAttempts = delays.length + 1;
let lastError: unknown;
@@ -37,7 +48,7 @@ export async function retry<T>(fn: () => Promise<T>, options: RetryOptions = {})
throw error;
}
const delay = delayMs * attempt;
const delay = delays[attempt - 1]!;
log.info(`» ${label} failed (attempt ${attempt}/${maxAttempts}), retrying in ${delay}ms...`);
await sleep(delay);
}
+12 -1
View File
@@ -88,8 +88,19 @@ async function dispatchFollowUpReReview(ctx: ToolContext, reviewedSha: string):
await ctx.octokit.rest.actions.createWorkflowDispatch({
owner: ctx.repo.owner,
repo: ctx.repo.name,
workflow_id: "pullfrog.yml",
workflow_id: getCurrentWorkflowFilename(),
ref: pr.data.base.repo.default_branch,
inputs: { prompt: JSON.stringify(payload) },
});
}
/**
* derive the running workflow's filename from `GITHUB_WORKFLOW_REF`, which has the form
* `<owner>/<repo>/.github/workflows/<filename>@<ref>` (e.g. `.../pullfrog.yaml@refs/heads/main`).
* falls back to `pullfrog.yml` if the env var is missing or malformed (shouldn't happen in CI).
*/
function getCurrentWorkflowFilename(): string {
const ref = process.env.GITHUB_WORKFLOW_REF ?? "";
const match = ref.match(/\/([^/]+)@/);
return match?.[1] ?? "pullfrog.yml";
}
+8 -8
View File
@@ -19,14 +19,14 @@ export async function handleAgentResult(ctx: HandleAgentResultParams): Promise<M
};
}
// Review and IncrementalReview modes intentionally never set wasUpdated:
// the prompt forbids report_progress (the review IS the durable record),
// and IncrementalReview's non-substantive path produces no review at
// all by design. wasUpdated staying false is also load-bearing for the
// stranded-comment cleanup in main.ts which deletes the "Leaping into
// action" orphan via `(!wasUpdated || trackerWasLastWriter)`. Skip the
// strict completion check for these modes — the agent's exit code is
// the completion signal, not a progress-comment write.
// IncrementalReview's non-substantive path exits cleanly without
// submitting any review, so no MCP write tool flips wasUpdated and the
// strict completion check below would otherwise fail the run. The
// isReviewMode skip is load-bearing for that path: the agent's exit
// code is the completion signal, not a progress-comment write.
// (Review mode that submits a real review now flips wasUpdated via
// create_pull_request_review, so the skip is redundant for the
// substantive-review path but kept for symmetry with IncrementalReview.)
// See plans/review_progress_comment_cleanup_b0120f6c.plan.md.
const mode = ctx.toolState.selectedMode;
const isReviewMode = mode === "Review" || mode === "IncrementalReview";
+74
View File
@@ -1,3 +1,4 @@
import { performance } from "node:perf_hooks";
import { describe, expect, it } from "vitest";
import { spawn } from "./subprocess.ts";
@@ -48,6 +49,79 @@ describe("spawn error path", () => {
expect(afterHandles).toBeLessThanOrEqual(beforeHandles);
});
it("killGroup: true propagates SIGKILL to grandchildren so close fires promptly", async () => {
// regression: node_modules/opencode-ai/bin/opencode is a Node shim that
// spawnSyncs the native binary with stdio:"inherit". without killGroup,
// child.kill("SIGKILL") hit only the shim — the native binary was
// reparented to PID 1, kept holding our stdout pipe via the inherited
// fds, and `child.on("close")` never fired (because pipes stayed open).
// a 5-min outer safety-net timer eventually rejected the agent promise,
// but the grandchild kept running until the GitHub Actions job-level
// timeout. this test replicates the shape with bash + a backgrounded
// sleep grandchild: with killGroup, close fires promptly after SIGKILL;
// without it, the parent would wait for sleep to exit (30s).
//
// the activity-check interval is fixed at 5s so the earliest the kill
// can fire is ~5s after start. budget 15s end-to-end.
const before = performance.now();
const result = await spawn({
cmd: "bash",
args: ["-c", "sleep 30 & wait"],
env: { PATH: process.env.PATH ?? "", HOME: process.env.HOME ?? "" },
activityTimeout: 1000,
killGroup: true,
}).catch((err) => err);
const elapsed = performance.now() - before;
expect(result).toBeInstanceOf(Error);
// 10s ceiling: 5s activity-check tick + signal delivery. a regression
// here (no killGroup) would hang for the full 30s sleep.
expect(elapsed).toBeLessThan(10_000);
}, 20_000);
it("isPausedExternally suspends the activity timer while it returns true", async () => {
// opencode's `task` tool encapsulates subagent execution in-process —
// subagent-internal events don't reach our parent NDJSON stream. this
// means the child's stdout falls silent for the full subagent duration
// even when real work is happening. opencode passes a predicate keyed
// off its in-flight task dispatch tracker; while it returns true, the
// activity timer must skip the kill decision and reset its baseline.
const result = await spawn({
cmd: "bash",
args: ["-c", "sleep 8; echo done"],
env: { PATH: process.env.PATH ?? "", HOME: process.env.HOME ?? "" },
activityTimeout: 1000,
isPausedExternally: () => true,
});
expect(result.exitCode).toBe(0);
expect(result.stdout).toContain("done");
}, 20_000);
it("isPausedExternally fires normally once it returns false again", async () => {
// verifies suspend/resume rather than unconditional bypass: the predicate
// flips to false after a few hundred ms, so the timer should resume from
// a fresh baseline (advanced during the paused window) and fire after
// activityTimeout idle time. critical that lastActivityTime is reset
// while paused — otherwise the resume tick would see a stale baseline
// and kill instantly.
let paused = true;
setTimeout(() => {
paused = false;
}, 500);
const result = await spawn({
cmd: "sleep",
args: ["30"],
env: { PATH: process.env.PATH ?? "", HOME: process.env.HOME ?? "" },
activityTimeout: 1000,
isPausedExternally: () => paused,
}).catch((err) => err);
expect(result).toBeInstanceOf(Error);
expect(String(result)).toMatch(/activity timeout/i);
}, 20_000);
it("reports signal-killed subprocesses as failures, not success", async () => {
// regression: before the fix, `child.on("close", (exitCode) => ...)`
// discarded the signal parameter and `exitCode || 0` coerced the
+54 -5
View File
@@ -106,6 +106,25 @@ export interface SpawnOptions {
stdio?: ("pipe" | "ignore" | "inherit")[];
onStdout?: (chunk: string) => void;
onStderr?: (chunk: string) => void;
// when true, spawn the child detached (its own process group) and route all
// kill paths (timeout, activity timeout, ctrl-c) through `process.kill(-pid, ...)`
// so signals reach grandchildren too. critical for binaries that fork through
// a shim (e.g. node_modules/opencode-ai/bin/opencode is a Node shim that
// spawnSync's the native binary; without killGroup, SIGKILL only hits the
// shim and the native binary is reparented to PID 1, holds our stdout pipe
// open, keeps emitting NDJSON, and `child.on("close")` never fires —
// producing zombie runs that hang until the GitHub Actions job timeout).
killGroup?: boolean;
// optional pause predicate consulted on every activity check. when it
// returns true the activity timer skips the kill decision *and* resets
// its idle baseline so a clean unpause can't immediately fire on a stale
// lastActivityTime. opencode uses this because its `task` tool encapsulates
// subagent execution in-process — subagent-internal events don't surface
// on the parent's NDJSON stream, so the local stdout-only signal would
// falsely fire mid-subagent. preferred over fake-activity heartbeats
// because there's no race window between a heartbeat tick and a subagent
// that finishes between ticks.
isPausedExternally?: () => boolean;
}
export interface SpawnResult {
@@ -127,6 +146,8 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
let stdoutBuffer = "";
let stderrBuffer = "";
const killGroup = options.killGroup ?? false;
return new Promise((resolve, reject) => {
// security: caller must provide complete env object, not merged with process.env
const child = nodeSpawn(options.cmd, options.args, {
@@ -136,10 +157,28 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
},
stdio: options.stdio || ["pipe", "pipe", "pipe"],
cwd: options.cwd || process.cwd(),
detached: killGroup,
});
// sends `signal` to the entire process group when killGroup is set, so
// grandchildren (e.g. the native opencode binary spawned by the
// opencode-ai Node shim) die with the parent. falls back to a direct
// child kill if the process-group send fails (common when the child
// already exited or was never made a process group leader).
const killSelf = (signal: NodeJS.Signals): void => {
if (killGroup && child.pid) {
try {
process.kill(-child.pid, signal);
return;
} catch {
// fall through to direct kill
}
}
child.kill(signal);
};
// track child for cleanup on Ctrl+C
trackChild({ child });
trackChild({ child, killGroup });
let timeoutId: NodeJS.Timeout | undefined;
let sigkillEscalatorId: NodeJS.Timeout | undefined;
@@ -157,7 +196,7 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
if (options.timeout) {
timeoutId = setTimeout(() => {
isTimedOut = true;
child.kill("SIGTERM");
killSelf("SIGTERM");
// track the escalator so a graceful SIGTERM response (close fires
// before the 5s elapses) can clear it. without capture, this timer
@@ -165,7 +204,7 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
// past a timed-out subprocess's clean exit.
sigkillEscalatorId = setTimeout(() => {
if (!child.killed) {
child.kill("SIGKILL");
killSelf("SIGKILL");
}
}, 5000);
}, options.timeout);
@@ -177,6 +216,16 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
`spawn activity timer: pid=${child.pid} cmd=${options.cmd} timeout=${activityTimeoutMs}ms`
);
activityCheckIntervalId = setInterval(() => {
// when an external pause predicate says we're suspended (e.g.
// opencode in a long-running task subagent whose internal events
// don't surface on stdout), advance lastActivityTime to "now" so a
// clean unpause doesn't immediately fire on a stale baseline, and
// skip the kill decision for this tick.
if (options.isPausedExternally?.()) {
lastActivityTime = performance.now();
log.debug(`spawn activity check: pid=${child.pid} paused externally`);
return;
}
const idleMs = performance.now() - lastActivityTime;
log.debug(
`spawn activity check: pid=${child.pid} idle=${Math.round(idleMs)}ms / ${activityTimeoutMs}ms`
@@ -186,9 +235,9 @@ export async function spawn(options: SpawnOptions): Promise<SpawnResult> {
killedAtIdleMs = idleMs;
const idleSec = Math.round(idleMs / 1000);
log.info(
`no output for ${idleSec}s from pid=${child.pid} (${options.cmd}), killing process`
`no output for ${idleSec}s from pid=${child.pid} (${options.cmd}), killing process${killGroup ? " group" : ""}`
);
child.kill("SIGKILL");
killSelf("SIGKILL");
clearInterval(activityCheckIntervalId);
try {
options.onActivityTimeout?.();