every 12h, scripts/find-newer-models.ts scans models.dev for newer GA
versions of every alias in action/models.ts and writes a focused
per-alias diff. .github/workflows/models-bump.yml short-circuits when
no candidates exist; otherwise hands the diff to pullfrog/pullfrog@main
to evaluate against the policy in wiki/model-resolution.md and open a
single living PR on the pullfrog/models-bump branch.
drops the brittle "latest model per provider" snapshot block in
action/test/models-catalog.main.test.ts (and its .snap file) — the cron
keeps the registry in sync with upstreams, and the remaining validity
tests act as the integrity gate on the bump PR.
* fix: don't reuse disabled proxy key on workflow re-runs; non-fatal title-gen errors
Three small surgical fixes addressing run https://github.com/pullfrog/app/actions/runs/25580969379:
1. **`/api/proxy-token` idempotency now checks `finalizedAt`.** GitHub re-runs
share the same `run_id` (only `run_attempt` increments), so attempt N+1's
action calls /api/proxy-token and inherits attempt N's `proxyKeyId`. The
`workflow_run.completed` webhook between attempts retires that key on
OpenRouter (`disableKey`), so attempt N+1 was getting back a disabled key
and OpenRouter responded with `401 User not found` on every call. Falling
through when finalized routes through the same billing gate
(`handleRouterBilling` balance check), so no new attack surface.
2. **OpenCode title-gen / small-model errors no longer fatal.** OpenCode
auto-spawns a small `agent=title small=true` background call at session
start to name the thread, defaulting to `anthropic/claude-haiku-4.5`
(anomalyco/opencode#1243). Pre-fix, the wrapper's `error` event handler
treated any `type=error` as fatal, so a cosmetic title failure killed the
run before primary inference even started. Now: stderr matching `small=true`
sets a one-shot suppression flag for the next stdout `error` event, which
is logged as a warning instead.
3. **Provider-error classifier puts auth patterns above rate-limit.** OpenRouter
401 payloads bundle `x-ratelimit-*` response headers, and the loose
`\brate[_ ]limit/i` pattern was winning. Added 401/403 status, `User not
found`, `Invalid authentication`, `No auth credentials found` patterns
ahead of rate-limit. Updated the existing 401-headers regression test to
assert correct auth classification rather than `null`.
* opencode: correlate small-model error suppression by message, not by next-event
Pullfrog self-review on #636 flagged a real concurrency hole. OpenCode forks
the title-gen call (`session/prompt.ts:1452-1457` via `Effect.forkIn(scope)`)
so it races primary inference. The previous one-shot `suppressNextErrorEvent`
boolean had no per-call correlation: it was consumed by whichever stdout
`type=error` event landed next, regardless of which subagent produced it.
Under concurrent failures, a primary-agent error landing first could be
silently downgraded to a warning while the small-model error then propagated
fatally — the inverse of the bug the suppression was meant to prevent.
Replaced the boolean with a `Set<string>` of pending small-model error
messages. stderr extracts the inner `"message":"..."` from any classified
provider error tagged `small=true`; the stdout `error` handler suppresses
only when `event.error.data.message` matches a pending entry. Set is capped
at 32 entries so a long stream of small-model failures can't wedge memory.
Also corrected the comment that referenced "session summarizer" — verified
in opencode source that summarize() does NOT use `small: true`; only the
title generator does today (only `small: true` match in the codebase).
* revert: drop opencode title-gen suppression
We have no evidence — and can't construct a realistic scenario — where
title-gen fails on an otherwise-successful run. Title-gen and primary share
the same OPENROUTER_API_KEY and hit the same proxy/upstream; whatever breaks
one breaks the other. The original repro on run 25580969379 is fully
explained by the stale proxy key (fix#1) — title-gen happened to be the
first call that surfaced the auth error, but every subsequent primary call
would have died the same way.
Suppression code adds complexity (cross-stream correlation logic, message
matching, set capping) and a real failure mode of its own (a small-model
error with a unique message could mask an unrelated primary error landing
shortly after). Net negative. Removing.
* opencode: surface subagent events via injected plugin
opencode's cli/cmd/run.ts event loop filters all message.part.updated
events to the orchestrator's session id (`part.sessionID !== sessionID`
continue), so subagent-internal tool_use / text / step events were
silently discarded by the CLI in --format json mode. opencode plugins,
by contrast, receive every bus event via bus.subscribeAll() regardless
of session.
ship a per-run plugin (action/agents/opencodePlugin.ts) that re-emits
non-orchestrator message.part.updated events as `pullfrog_bus_event`
envelopes on opencode's stdout. the plugin is staged into
<XDG_CONFIG_HOME>/opencode/plugin/pullfrog-events.ts which is already
redirected to ctx.tmpdir — never the user's repo working tree.
the plugin also forwards the orchestrator's task tool dispatch at
state.status="running" — that's the first moment state.input is
populated with description / subagent_type / prompt and it lands
BEFORE the subagent's first message.part.updated. forwarding this
lets SessionLabeler register the lens label early, so subagent
events bind to the correct lens name (e.g. lens:correctness) instead
of the subagent#N fallback. the existing tool_use handler dedupes
on callID so the late status=completed event from the CLI doesn't
double-record.
the parent's pullfrog_bus_event handler synthesizes the equivalent
CLI-style event for each part type (tool/step-start/step-finish/text)
and dispatches through the same handlers used by orchestrator events,
so labeling, tool-call rendering, and the formatWithLabel magenta
prefix all share one code path.
verified end-to-end via `pnpm play --local --raw` with a prompt that
dispatches a reviewfrog subagent: orchestrator's task call now logs
"» dispatching subagent: lens:read-readme-and-report-purpose" before
the subagent runs, the subagent's read tool call surfaces with
[lens:...] magenta prefix, and the run-end "subagent finished"
attribution shows the lens name.
also adds an AGENTS.md rule formalizing the no-write-to-repo
invariant: action runtime must never write into the user's working
tree; auxiliary files go in ctx.tmpdir via HOME / XDG_CONFIG_HOME.
* drop opencodePlugin.test.ts — bullshit-test cleanup
these tests spied on process.stdout.write, loaded the plugin source
into a temp file via dynamic import, and asserted the output strings
matched the plugin source i'd just hand-written. zero unique signal
over the e2e run in preview repo, plus they violate AGENTS.md's
"mocks tend to add ceremony and brittleness" rule. real signal lives
in the e2e: lens label rendering, dispatch attribution, no double
events. if a syntactic regression in the plugin source ever ships,
opencode logs it on plugin load and the e2e fails fast — the unit
tests would catch the same regression no faster.
* remove isPausedExternally — plugin makes it unnecessary
empirical proof from PR #634's e2e debug trace: ~3.3 pullfrog_bus_event
lines per second arrive on the parent's child.stdout pipe during a
typical subagent run. each one fires updateActivity() and resets
lastActivityTime, so the inner spawn activity timer naturally stays
armed-but-not-fired throughout the subagent's lifetime — no suspend
predicate needed.
drop:
- SpawnOptions.isPausedExternally + the check in spawn()'s activity loop
- isSubagentInFlight() in opencode.ts + its callsite
- two isPausedExternally unit tests in subprocess.test.ts
keep:
- killGroup (the actual zombie-prevention fix; still tested)
- the plugin (action/agents/opencodePlugin.ts; the architectural fix)
- everything in opencode.ts that derives lens labels from task dispatches
the only edge case isPausedExternally covered that the plugin doesn't
is a non-streaming provider going silent for >5min during a single
LLM call inside a subagent. that's a provider-behavior question, not
a harness-architecture one — best fixed at the provider level if it
shows up. defense-in-depth that adds indirection is harmful when the
upstream architectural fix is already in place.
* opencode: address review feedback on bus envelope routing
three findings from PR #634 review (2026-05-08T22:13:44Z):
1. token/cost double-count: routing subagent step_finish through the
orchestrator's handler folded subagent tokens/cost into the run-wide
accumulators that flow to logTokenTable + AgentUsage. neighbouring
init/text handlers all gate on ORCHESTRATOR_LABEL for exactly this
reason. fix: drop step_start AND step_finish from the bus envelope
handler — those carry orchestrator-scoped state (currentStepId,
stepHistory, token accumulators) that subagent events shouldn't
touch. tool calls and text from subagents still surface — that's
the user-visible activity.
2. subagent tool errors invisible: routed status="error" tool parts
into handlers.tool_use which only emits "» <tool>(...)" with no
error indication. fix: extend handlers.tool_use itself to log
"» tool call failed: <msg>" when state.status==="error". benefits
the orchestrator path too — opencode CLI also emits failed tool
calls as tool_use at status=error and we were swallowing the
failure signal there as well.
3. stale comments + leaked local paths: plugin source had
/tmp/opencode-investigate/... paths from my local clone, specific
line numbers from opencode's dev branch that don't match v1.1.56,
forkDetach claim that's wrong for the pinned version, and JSDoc
that still listed message.updated/session.error in the forwarded
set after the runtime filter narrowed to message.part.updated only.
fix: drop machine-local paths, drop version-fragile line numbers,
correct the forwarded-set list, generalize the
"why no @opencode-ai/plugin import" rationale to be version-agnostic.
second review (2026-05-08T22:27:58Z) confirms these are the only
findings still open — no new issues from the isPausedExternally
removal.
* learnings: edit-in-place tmpfile (drop update_learnings tool)
learnings now follow the PR-summary file pattern: server seeds
`pullfrog-learnings.md` from `Repo.learnings` at startup, agent reads
it as part of context, may edit in place during the post-run reflection
turn, server reads back at end-of-run and PATCHes if changed.
motivation: `update_learnings` required the agent to pass the FULL
merged list as a string parameter — an output-token tax that grew
linearly with the learnings size, and a constant prompt-context
expansion since the contents were also inlined into the LEARNINGS
section. for repos with mature learnings the prompt was getting
visibly noisy in CI logs.
key changes:
- new `action/utils/learnings.ts` (seed/read helpers + 10k cap)
- `main.ts`: always seed; `persistLearnings` mirrors `persistSummary`
(success path, error path, exit-signal handler, idempotent guard,
byte-trim equality skip); forwards `model` for `LearningsRevision.model`
- `LEARNINGS` prompt section now contains only the file path + a
one-line "read it" instruction (no contents inlined)
- `update_learnings` MCP tool deleted; `action/mcp/learnings.ts` removed
- reflection turn (`buildLearningsReflectionPrompt`) reframed around
file editing with explicit prune-stale + leave-alone-if-nothing-new
framing
- `learningsStep` removed from every mode checklist — surface lives only
in the LEARNINGS prompt section + the reflection turn now
* learnings: harden seed step + refresh stale docs (review feedback)
Three findings from PR review, all implemented:
1. wrap learnings seed in best-effort try/catch (action/main.ts) —
the always-on seed block ran unconditionally and an unwrapped
`seedLearningsFile` (mkdir + writeFile) failure (ENOSPC, EACCES,
hostile sandbox) would unwind into the outer main() catch and flip
an otherwise-successful run to "❌ Pullfrog failed" before the
agent even started. asymmetric with `persistLearnings`'s own
best-effort contract. wrap and log on failure; downstream
consumers (`persistLearnings`, agent harnesses, `resolveInstructions`)
already handle `learningsFilePath: undefined` cleanly.
2. refresh wiki/main.md — `resolveInstructions` parameter renamed
from `learnings` to `learningsFilePath` in this PR; the data-flow
diagram and the resolver dependency table both still showed the
pre-refactor signature.
3. drop deleted `learnings.ts` from ROADMAP.md + RESEARCH.md
"missing MCP tool tests" bullets — `action/mcp/learnings.ts` was
removed in this PR; the bullets are otherwise still accurate.
makes debugging easier by emitting a single `» <verb> <kind> <id>` line
after every successful GitHub write (and upload) the agent performs via
the Pullfrog MCP, mirroring the chevron convention used elsewhere.
* spawn: kill process group + heartbeat subagent activity
two compounding bugs produced zombie agent runs that stalled until the
GitHub-Actions job-level timeout (observed on PR #622, run 25577068620).
1. SIGKILL hit the wrong process. node_modules/opencode-ai/bin/opencode
is a Node shim that spawnSyncs the native opencode-<plat>-<arch>
binary with stdio:"inherit". our spawn() ran without detached, so
child.kill("SIGKILL") killed only the shim. the native binary was
reparented to PID 1, kept holding our stdout pipe via inherited fds,
and child.on("close") never fired — leaving the agent promise
pending past the 5min outer safety-net timer ("agent still pending
5min after inner activity kill — forcing exit") and the grandchild
running until the runner timed out.
fix: SpawnOptions gains killGroup; when set, we spawn detached and
route all kill paths (timeout, activity timeout, ctrl-c) through
process.kill(-pid, signal). opencode + claude opt in.
2. inner activity timer false-fired during long task subagents.
opencode's `task` tool encapsulates subagent execution in-process —
subagent-internal events don't reach the parent NDJSON stream — so
the parent looked idle for the full subagent duration even when
real work was happening, and the 5min DEFAULT_ACTIVITY_TIMEOUT_MS
would fire mid-subagent.
fix: SpawnOptions gains externalActivitySource; the timer fires on
min(local stdout idle, external idle). opencode passes getIdleMs()
from the global activity tracker and runs a 30s heartbeat
(markActivity()) while at least one task dispatch is in flight.
action/utils/subprocess.test.ts covers both: a bash+sleep grandchild
that proves close fires <10s with killGroup, and externalActivitySource
keeping the timer armed during 8s of stdout silence.
* opencode: suspend activity timer instead of heartbeat during subagent runs
addresses review on prior commit: replace the 30s markActivity()
heartbeat with a boolean isPausedExternally predicate keyed off
opencode's existing taskDispatchByCallID + pendingTaskDispatches.
no fake activity, no race window between a 30s tick and a subagent
that finishes between ticks.
while the predicate returns true, spawn's activity check skips the
kill decision *and* advances lastActivityTime so a clean unpause
can't fire on a stale baseline. tests cover both the suspended case
(8s of stdout silence + activityTimeout=1s but paused → process
exits cleanly) and the resume case (paused for 500ms then unpaused
→ 30s sleep gets killed by activity timeout as normal).
Three real defects flagged in the post-merge review of #616, plus one cheap
hardening:
1. OpenCode `limit.output` override was a silent no-op on opencode-ai@1.1.56.
Top-level `limit.output` has no read site in OpenCode (verified against
the v1.1.56 source: `OUTPUT_TOKEN_MAX = Flag.OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX
|| 32_000` in session/llm.ts; per-model `model.limit.output` has its own
scope). Plumbed via `OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX=5000` env var
on the OpenCode spawn instead. Drops dead `OpenCodeConfig.limit?` type
field and the corresponding config write in `buildSecurityConfig`. This
was the headline mechanism of #616 — without the env var, the upfront
`max_tokens` reservation stayed at 32_000 and low-wallet runs continued
failing the way #616 was supposed to prevent.
2. Phantom auto-reload buffer for detached-card accounts. DELETE
/payment-method clears `stripeCustomerId` but leaves `autoReloadEnabled`
intact, so an account with welcome-credit residue and a detached card
could mint a key with `keyLimitCents = balance + autoReloadAmountCents`
($50 default, schema-cap $100K) of free spend headroom we have no way
to bill. Conjunctive `account.autoReloadEnabled && hasCard` in the
buffer selection closes this. Defense-in-depth follow-up worth doing:
clear `autoReloadEnabled` in the card-detach handler.
3. The autoReloadEnabled 402 branch fired for phase-1 noop paths
(`!stripeCustomerId`, `reloadAmountCents < 50`, `balance >= threshold`)
where `result.failure == null`, returning `"insufficient balance"` with
no actionable code. Gated on `result.status === "failed"` so non-charge
paths fall through to the `hasCard` / no-card branches and emit
`router_balance_exhausted` / `router_requires_card` instead.
4. (cheap) `ROUTER_KEYLIMIT_EXHAUSTED_PATTERN` now uses `/is` instead of
`/i` so `.*?` crosses newlines. Defends the BillingError reclassification
against any upstream layer that wraps the OpenRouter error onto multiple
lines. Trivial.
Test plan: 488/488 unit tests pass (1 new test for newline regex behavior).
`toolState.model` was set only to `payload.model` (the stored slug, often
undefined for router/oss runs that derive the target from `proxyModel`).
the footer's "Using `…`" segment is gated on a truthy model, so router
runs on repos without an explicit model setting shipped reviews/comments
with no model badge — e.g. PR #614's review showed no model despite
running `openrouter/anthropic/claude-opus-4.7` via proxy.
now mirror the priority used by `resolveModelForLog` and `isGeminiRouted`:
`payload.proxyModel ?? resolvedModel ?? payload.model`. also reverse-look
up by `resolve`/`openRouterResolve` in `formatModelLabel` so a proxy
target like "openrouter/anthropic/claude-opus-4.7" still renders as
"Claude Opus".
Replaces today's `keyLimitUsd = min(walletBalance, $25)` with population-aware
buffers so users can use 100% of their credits before being paywalled, and
opaque mid-run "more credits" failures (e.g. https://github.com/pullfrog/app/actions/runs/25531633203)
get a clear PR comment instead of a generic stack-trace dump.
Policy matrix:
- Auto-reload accounts: `wallet + autoReloadAmountCents` (default $50, no cap)
- Card + no-autoreload: `wallet + $5` overdraft buffer
- No card: `wallet` (no buffer; existing zero-balance 402 stays)
- OSS: `$10` (unchanged)
Removes the $25 per-run cap entirely. Long Build runs at high-balance
accounts no longer silently cap at $25.
Other changes:
- Classify mid-run OpenRouter "requires more credits, or fewer max_tokens"
errors as `router_keylimit_exhausted` BillingError so users get an
actionable PR comment.
- Override OpenCode `max_tokens: 32000` default to `5000` via
OpenCodeConfig.limit.output. Drops Opus per-call upfront budget reservation
from ~$2.40 to ~$0.38 — what makes low-wallet runs viable at all.
- Switch `findInitialComment` and `findExistingPaywallComment` to GraphQL
`issueOrPullRequest(number:) { comments(last: 100) }` (single round trip,
actually returns newest-100; REST listComments doesn't support sort/direction).
Also fixes a latent `comments.find()` returning the OLDEST match instead
of the most recent — now selects max(databaseId).
- Wrap `syncAccountUsage` in `prisma.$transaction` with `SELECT ... FOR UPDATE`
on the account row. Pre/post-balance reads inside the transaction enable
deterministic low-balance edge detection (currently logs; will push the
outreach.low_balance task once #592 lands).
Plan: .cursor/plans/router-low-balance-paywall.plan.md (in companion wiki-billing branch)
Was retained on `workflow_runs` after PR #568 replaced the comment-based
summary path with the snapshot architecture, with a "kept for backfill of
pre-snapshot runs" annotation. No backfill is planned: pre-snapshot summary
comments were written in the user-facing PR_SUMMARY_FORMAT (TL;DR + key
changes blockquote + before/after sections), not the agent-context
functional-summary format the snapshot now expects. Backfilling them would
prime new runs with the wrong shape and pollute the agent context. Old
comments stay on github.com as historical artifacts; the column on the DB
row is dead weight.
Strips the field from:
- prisma schema + new migration `20260508190000_drop_summary_comment_node_id`
- `app/api/workflow-run/[runId]/route.ts` STRING_FIELDS allowlist
- `action/utils/patchWorkflowRunFields.ts` type union + STRING_KEYS
- `utils/db/selectActiveWorkflowRuns.ts` select clause
- `utils/github/enrichWorkflowRunsWithArtifactUrls.ts` node-id type, URL
resolution, collectUniqueNodeIds + urlsForRun
- `utils/webhooks/handleWorkflowRunWebhook.ts` two select clauses, the
hasRecordedArtifact param, and the orphaned-leaping-comment alert text
- `components/RunArtifactPills.tsx` ArtifactKey union + ARTIFACT_KEYS +
switch cases (drops the "View summary" chip from the workflow run list)
Verified: pnpm typecheck clean, pnpm lint clean (537 files), action build
clean. Dev DB reset against production parent and the migration applied
cleanly — column is gone from the workflow_runs table.
* PR summary as agent-edited tmpfile snapshot
Replaces the comment-based PR summary path (and the in-progress
update_pr_summary tool from #534) with a snapshot file the agent edits in
place during Review / IncrementalReview / pr-summary Task runs.
The server seeds the tmpfile with the previous snapshot (incremental) or a
stable scaffold (first run), exposes the path via select_mode, and reads it
back at end-of-run to persist to WorkflowRun.summarySnapshot and (when the
prSummaryComment toggle is on) splice into the PR description body.
Why a tmpfile rather than a tool call: incremental snapshot edits are
output-token-cheap when the agent uses native file-editing tools, and
range-diff cleanly across runs because section headings are stable. The
agent never has to regurgitate the full snapshot to update it.
Gating: snapshot generation is opt-in via either prSummaryComment="enabled"
(splice into PR body) or prReReview="enabled" (snapshot feeds future
incremental review runs as context). Users who disable both pay nothing
end-to-end — no seeding, DB write, or body splice.
Behavior changes:
- Drop the Summarize mode and the Summary comment type entirely; the
rolling summary is no longer a separate run shape.
- pull_request_synchronize with re-review off and summary on still
dispatches a silent pr-summary Task, but it edits the snapshot file
instead of posting a fresh comment.
- /api/repo/.../pr/.../summary-comment now returns
{ snapshot: string | null } from the DB instead of fetching a comment via
GraphQL. URL kept stable so deployed older actions degrade gracefully.
- summaryCommentNodeId is retained on WorkflowRun for legacy data and a
future backfill of pre-snapshot comment-based summaries.
Supersedes #534. The commit-tool/sub-agent direction in that PR is
abandoned in favor of this file-based shape.
* address review pass #1: synchronize fallback, splice idempotency, docs
* address review pass #2: in-flight skip should not race summary fallback
* address review pass #3: signal-handler flush, doc clarifications
* address review pass #4: in-flight persist promise + bounded body-splice timeout
* address review pass #5: defensive catch on persist worker, doc nit
* add summary-stale post-run gate
When generateSummary is set, we capture the bytes of the seeded snapshot
file and pass them to the agent's post-run loop alongside the file path.
After each agent attempt, the loop diffs the current file against the
seed; if they're byte-identical the agent never touched it, and we nudge
once via a resume turn (similar to the dirty-tree gate, but soft and
fire-once so smaller models that legitimately decide no edit is warranted
don't burn the retry budget).
Mostly defends against forgetful smaller models on the Review path —
their mode prompt asks them to edit the snapshot file, but the
multi-step instruction can fall through when the diff is large.
* trigger: retry vercel preview build
* fix(action): drop unused re-export that pulled node:fs/promises into next bundle
action/internal/index.ts was re-exporting DEFAULT_PR_SUMMARY_INSTRUCTIONS
from action/utils/prSummary.ts, but nothing in the next.js app imports
it. prSummary.ts uses node:fs/promises, and pullfrog/internal is aliased
into the next bundle by next.config.ts, which made turbopack try to
resolve node:fs/promises in client chunks and fail with:
the chunking context (unknown) does not support external modules
(request: node:fs/promises)
drop the re-export — selectMode.ts (the only real consumer) already
imports it directly from action/utils/prSummary.ts.
* firewall PR summary snapshot from user instructions; resurrect rich format for Review
The agent-internal snapshot (the markdown file the agent edits in place across
runs) is exclusively durable context for future agent runs — user-supplied
summarization instructions warp it and degrade that context. Drop the
prSummaryCommentInstructions read path end-to-end:
- handleWebhook: stop reading prSummaryCommentInstructions, stop passing
prSummaryInstructions through dispatch options
- action payload + ToolState + selectMode addendum: drop the instructions
appendix; the snapshot prompt is fixed, not user-shaped
- TriggersSettings: drop the InstructionsEditor for prSummaryCommentInstructions
- prSummary.ts: reframe DEFAULT_PR_SUMMARY_INSTRUCTIONS as agent-targeted
(durable context, not human-facing prose)
Prisma columns (prSummaryComment, prSummaryCommentInstructions) and the
matching zod schema entry stay for graceful retreat.
Separately, resurrect PR_SUMMARY_FORMAT (deleted along with the Summarize mode
in the original PR) and wire it into Review mode only. Initial PR reviews now
include a structured summary section in the review body using the rich format
(TL;DR, key changes, ## sections with before/after, file-link trails).
IncrementalReview keeps its existing terser bullet-list shape since re-review
bodies are deltas, not introductions. The user-facing review summary and the
agent-internal snapshot are deliberately separate artifacts with separate
prompts and zero shared content.
* address review comments: prompt self-consistency + stale-doc cleanup
PR 568 self-review (4232488109) flagged a self-contradiction the firewall
commit introduced and three stale doc references that survived.
- action/modes.ts: Review-mode step 2's trivial-PR shortcut said `submit
"Reviewed — no issues found." per step 5`, but step 5's rewrite removed
exactly that preamble. Aligned both: trivial PRs and no-actionable-issues
PRs now produce a body that opens with "No new issues found." followed by
the PR summary, so the user gets the headline up front and still sees what
was reviewed.
- docs/pr-reviews.mdx: dropped the "customize the summary style with Summary
instructions in the console" sentence (the editor was removed in the
firewall commit). Replaced with a note that the snapshot uses Pullfrog's
built-in format and is not user-customizable.
- wiki/prompt.md, wiki/modes.md: rewrote the snapshot-prompt entries to
reflect the firewall — DEFAULT_PR_SUMMARY_INSTRUCTIONS is the entire
prompt, prSummaryCommentInstructions is no longer wired in.
* drop orphaned prSummaryCommentInstructions column
Prod audit (455 repos): 5 non-null rows on a single account, all containing the
literal placeholder text from the InstructionsEditor we removed in the firewall
commit. No account has an intentional preference set, so silent-ignore (the
keep-for-retreat option) costs us nothing meaningful while leaving an orphan
column in the schema. Drop it.
- prisma/schema.prisma: remove the column
- prisma/migrations/20260506000000_drop_pr_summary_comment_instructions:
ALTER TABLE ... DROP COLUMN
- utils/schemas/triggers.ts: drop the matching zod entry
* drop body splicing; snapshot is internal-only
User-visible PR summarization continues to ship in Review and IncrementalReview
review bodies (which already render PR_SUMMARY_FORMAT and "Reviewed changes"
respectively). The snapshot tmpfile is now purely durable cross-run agent
context — seed, edit-in-place, save to DB, feed the next run. Massive
simplification: the body splice mechanics, the two-toggle gating matrix, the
summaryHandlingCovered race tracking, and the synchronize summary-only Task
fallback all go away.
Code:
- prSummary.ts: drop splice/strip/marker code (`splicePrSummary`,
`stripExistingSummaryBlock`, `buildSummaryBlock`, `extractPrSummary`,
PULLFROG_SUMMARY_START/END). keep scaffold, instructions, seed/read.
- main.ts: rename persistAndPostSummary -> persistSummary; collapse to a
single DB PATCH. drop pulls.get/pulls.update, drop AbortSignal timeout,
drop in-flight promise machinery, drop prSummaryToBody plumbing.
- ToolState: add summarySeed (replaces local var in main.ts so persist can
compare). drop prSummaryToBody and summaryPersistInFlight.
- persistSummary now compares against the seed and skips the DB write
with a warning when unchanged — saving the seed verbatim is either a
no-op or persists the placeholder scaffold, neither useful.
- postRun.ts: when summary-stale is the only failing gate and the resume
turn itself fails, restore the pre-resume successful result and break.
symmetric with the existing reflection-failure preservation. summary-stale
can no longer flip a successful run to failed.
Webhook:
- pull_request_opened: generateSummary follows prReReview only (the snapshot
has no consumer when re-review is off).
- pull_request_synchronize: collapses to "if prReReview enabled, dispatch
IncrementalReview". the summaryHandlingCovered flag, the same-SHA/in-flight
coordination it was protecting, and the summary-only Task fallback all
delete cleanly.
UI / config:
- drop SummarizePRsTrigger (the toggle gated body splice; with that gone
it has no behavior). drop sidebar entry, console import, Text icon import.
- drop prSummaryComment from triggers zod schema, prisma schema, preview
settings script.
Migration: squash the two existing migrations into one timestamped
20260507000000_pr_summary_snapshot covering all three column changes
(add summarySnapshot on workflow_runs, drop prSummaryCommentInstructions
and prSummaryComment on repos). repo convention is one migration per PR.
Action: bump 0.0.203 -> 0.0.205 (payload contract changed: prSummaryToBody
removed; main is at 0.0.204).
Out-of-diff cleanup:
- review.ts:190 + review.test.ts:651 — "Reviewed — no issues found." ->
"No new issues found." to match the canonical body in modes.ts.
Verified: pnpm typecheck clean, pnpm lint clean, postRun + review tests
pass, dev DB reset against production and the squashed migration applied
cleanly (summarySnapshot present, prSummaryComment / prSummaryCommentInstructions
both gone).
* re-orient snapshot toward functional summary; drop prior-review-feedback section
Empirical audit on preview-568 PR #5 showed the snapshot IS load-bearing
for the orchestrator: lens-dispatch prompts on incremental runs carried
forward context from the snapshot's risk register (e.g. "the JSDoc
explicitly scopes to code points — do not flag grapheme-cluster issues"
on the surrogate-pair fix run, "consistency with native padStart" on the
padStart-added run). The orchestrator was reading the snapshot, reasoning
about it, and using it to anti-prime / focus subagents — exactly the
high-leverage path. My earlier "snapshot is write-only" claim was wrong.
The shape, however, was steering it toward review-history-log instead of
functional summary. This commit re-orients:
- prSummary.ts: replace the four-section scaffold (~580 chars of placeholder
italics under "What this PR does / Key changes / Risk / Reviewed in prior
runs") with a minimal seed (~150 chars: just a header + a one-line
comment about what the file is for). different PRs warrant different
organization; forcing a refactor and a feature into the same template
is procrustean. minimal seed also makes the unchanged-from-seed gate
in persistSummary more sensitive.
- selectMode.ts addendum: rewrite around three principles. (1) the snapshot
is a FUNCTIONAL summary of what the PR does and the risks it carries,
not a chronological review log — commit history can already be
reconstructed from list_pull_request_reviews. (2) the orchestrator should
USE the snapshot during triage and dispatch — concrete example given of
carrying snapshot context into subagent lens prompts. (3) structure is
the agent's call; stable headings make snapshots range-diff cleanly when
they fit, but riff when they don't.
- modes.ts IncrementalReview: drop the "Prior review feedback" checklist
from the user-facing review body (step 6b gone, step 7 ELSE IFs cleaned
up). It duplicated content that's already covered by the Reviewed-changes
bullets and tracked durably in the snapshot for the next agent run; in
the user-facing body it was noise. step 3 still fetches prior reviews
but its role is now just filtering aggregation in step 5, not rendering.
- AGENTS.md: codify "no follow-ups" rule. when an issue is identified
during code review, fix it in this PR — PR scope does not constrain
quality. follow-up TODOs are forbidden as a substitute for doing the
work now.
Empirical evidence supporting the re-orientation:
- Run 25568912293 (PR#5 incr1, surrogate-pair fix): orchestrator's
correctness lens dispatch said "Do NOT flag grapheme-cluster issues
— the JSDoc scopes to code points." The grapheme-cluster framing was
not in the diff; it was downstream of the snapshot's prior risk-section
framing of truncate's contract. Snapshot influencing dispatch.
- Run 25569054779 (PR#5 incr2, padStart added): orchestrator's correctness
lens dispatch enumerated edge cases including "consistency with native
String.prototype.padStart contract" and "fill = multi-code-point string
(e.g. emoji)". Both threads carried over from the snapshot's prior
truncate code-point-vs-code-unit discussion. Snapshot informing the
shape of what was looked for.
The cost of maintaining the snapshot (~800 tokens, ~$0.005/run) is
trivially affordable when it materially improves orchestrator triage
on the 1-5 lenses dispatched per review.
* action: quieter, deep-linked billing error comments
The PR progress comment for billing errors led with a loud `### ❌
Pullfrog billing error` H3 and pointed at the bare `/console` index page
regardless of which org owned the repo. Make the copy quieter and more
actionable:
- bold first line instead of an H3 (the comment already has Pullfrog
branding in the footer, no need for a second header)
- thread `runContext.repo.owner` into the formatters and deep-link to
`pullfrog.com/console/<owner>#billing` (or `#model-access` for the
router-needs-card branch)
- split the old "insufficient balance" default into two branches: card
declined (Stripe returned a declineCode — "we'll retry next run") vs.
balance empty (no in-flight charge — "top up or enable auto-reload")
- strip UX framing and pullfrog.com URLs from the proxy-token 402
responses; they're now terse signal-only strings, with all copy and
links rendered by the action so there's a single source of truth
* proxy-token: return 503 on phase-1 txn failure, not 402
Phase-1 only fails on server-side issues (serializable retry exhaustion,
Prisma/DB flake) — no Stripe call has happened yet, so it's not a
billing decline. Pre-PR this rendered as the generic "billing error —
manage billing" copy, which was vague-but-not-wrong; under the new
copy it would falsely tell the user their balance is empty.
Returning 503 routes the action through TransientError ("temporarily
unavailable, retry") which is the accurate framing.
Caught by Pullfrog review on PR #600.
* action: minimize pullfrog.yml permissions and drop actions:read
The recommended pullfrog.yml workflow asked for a permissions block that's
broader than what the action actually uses with the workflow GITHUB_TOKEN —
all real work (git push, PR comments, reviews) goes through installation
tokens that the action mints via OIDC. Customer security scanners flagged
the workflow-level block as too permissive.
- Move permissions to the job level and reduce to id-token: write,
pull-requests: write, issues: write. contents:read is the implicit default
and covers actions/checkout; contents:write, checks:read are unused by
any GITHUB_TOKEN consumer; actions:read was only used by post-cleanup's
listJobsForWorkflowRun call.
- Replace listJobsForWorkflowRun with a SIGTERM/SIGINT handler in main.ts
that calls core.saveState("cancelled", "true"); post-cleanup reads it
back via core.getState. Same cancel-vs-failure UX, no extra scope needed.
- Sync the docs (headless-action, getting-started, action/README) and the
two dogfood pullfrog.yml workflows to the new minimal block. Update the
post-cleanup wiki to describe the saveState approach.
* action: drop pull-requests/issues from required workflow scopes
Switch postCleanup.ts to mint its own short-lived installation token via OIDC
(acquireNewToken with issues:write + pull_requests:write) instead of using the
workflow GITHUB_TOKEN. Same comment-update behavior, but the workflow no longer
needs those scopes — the only permissions Pullfrog ever asks for are id-token:write
(OIDC exchange) and contents:read (actions/checkout).
Also fixes a bug from the previous commit: setting an explicit permissions block
drops every unlisted scope to none (with metadata as the only exception), so
omitting contents would have broken actions/checkout. Restored at both workflow
and job level.
* action: scope id-token:write to pullfrog job, not workflow level
id-token:write is the powerful one — it lets a job mint OIDC tokens that can
be exchanged for cloud credentials or our installation tokens. Keeping it at
workflow level means any future job added to this file silently inherits it.
Move it to the job level where it's actually used; leave only contents:read
at workflow level as a safe baseline for any future jobs.
* action: move stuck-comment cleanup server-side, drop write perms entirely
The action's post-cleanup step lived inside the runner and used the workflow
GITHUB_TOKEN to update the "Leaping into action…" progress comment when a run
failed/cancelled, requiring pull-requests:write + issues:write at the workflow
level. Move that responsibility to the workflow_run.completed webhook handler:
it already has installation-token access via the GitHub App, runs server-side
(no Pullfrog API dependency loop on failure), and lets us drop both write perms.
Recommended workflow permissions block is now truly minimal:
permissions:
contents: read
jobs:
pullfrog:
permissions:
id-token: write
contents: read
Server side
- handleWorkflowRunCompleted: when conclusion != "success" and the WorkflowRun
has progressCommentId, mint installation octokit and update the stuck comment
in place. Try issues.getComment first, fall back to pulls.getReviewComment on
404 (we don't store comment type — one wasted GET on the rarer review case).
- Reuses buildPullfrogFooter and updateProgressComment from pullfrog/internal,
matching the wording the action used to write client-side.
Client side
- Delete action/utils/postCleanup.ts and action/post.ts.
- Remove post: + post-if: from action/action.yml.
- Drop runPostCleanup wiring from action/commands/gha.ts and action/play.ts.
- Remove the SIGTERM/saveState handler I added in main.ts in the previous commit
(no longer needed; cancel/fail signal comes from the webhook hook payload).
Plumbing
- Extract isLeapingIntoActionCommentBody into action/utils/leapingComment.ts so
the predicate can be re-exported via pullfrog/internal without dragging the
MCP server's transitive type graph into the Next.js app's typecheck.
- mcp/comment.ts re-exports from the new location for backward compat.
Wiki
- Delete wiki/post-cleanup.md (obsolete; cleanup is now a one-liner branch in
the workflow_run webhook handler).
* chore: ignore .worktrees in biome config
Recently-added pnpm worktree feature creates nested git worktrees under
.worktrees/, each with their own biome.jsonc declaring root. Biome's
recursive scan trips on the nested config and fails pnpm lint. Excluding
the directory matches the existing .gitignore entry.
* fix: address PR #594 review findings
Two real bugs caught by code review:
1. handleWorkflowRunWebhook.ts:323 — drop the /m flag on the stuck-comment
detection regex. With /m, ^ matches any line start, so any finalized
progress comment that embeds a task list (report_progress writes
`- [x]`/`- [ ]` lines via todoTracking.ts) would be flagged as "stuck"
and silently overwritten with the "This run croaked" boilerplate
whenever the workflow concluded non-success after the agent's final
summary already landed. Restores the body-start anchoring the original
in-process postCleanup.ts:90 had.
2. action/scripts/check-entrypoint-imports.ts — drop ../post.ts from the
esbuild entry-point list (the file was deleted in aa43b9af). The
`pnpm check:entrypoints` step in test.yml would have failed on every
run with an unresolvable-entry-point error.
Plus three small follow-ups:
- main.ts:580 — comment said "post-cleanup has its own verify-retry loop"
but post-cleanup is gone. Updated to describe the new server-side path.
- mcp/comment.ts:443 — comment said "so post script doesn't think the run
failed". Updated to describe the actual current consumers of wasUpdated.
- commands/gha.ts:84 — `--post` help text said "run post-cleanup flow" but
with the post-cleanup path removed, --post is only valid alongside the
`token` subcommand for installation-token revocation. Updated wording.
* fix(action): scope --post help text to gha token subcommand
Root gha help text was documenting --post, but --post only makes sense
paired with the token subcommand (it's how the post step revokes the
installation token previously acquired in the main step). Move it to a
dedicated gha token help section and add a parser layer that rejects
--post on the bare gha command.
$ pullfrog gha --help
usage: pullfrog gha [subcommand]
...
options:
-h, --help show help
$ pullfrog gha token --help
usage: pullfrog gha token [--post]
...
options:
-h, --help show help
--post revoke the previously-acquired token (post-step usage only)
* webhook: artifact-aware cleanup of stranded leaping comments on success
Previously the workflow_run.completed cleanup only handled non-success
conclusions. Extend it to also catch the rare case where a successful
run leaves a "Leaping into action…" comment stuck (in-process cleanup at
action/main.ts:723 normally handles this, but can be skipped on SIGKILL,
runner host crash, or any exit path that bypasses main()'s finally block).
New behavior in cleanupStuckProgressComment:
- cancelled → update with "cancelled 🛑" body (unchanged)
- failure (other) → update with "croaked 😵" body (unchanged)
- success + artifact recorded → delete the comment (the artifact is the
user-facing surface; the leaping comment
is just stale UI noise at this point)
- success + no artifact recorded → delete the comment AND alert
team@pullfrog.com via emailAlert
The "success + no artifact" path is "should never happen" territory: the
run claims success but produced no review, PR, issue, plan, or summary
comment. The team alert helps us catch in-process cleanup regressions or
artifact-tracking gaps. hasRecordedArtifact reads {review,pr,issue,
planComment,summaryComment}NodeId off the WorkflowRun row to make the call.
* webhook: narrow stuck-comment detection to leaping prefix only
Drop the stranded-todo-pattern branch from cleanupStuckProgressComment.
The leaping prefix is highly specific and impossible to confuse with a
legitimate summary; a leading todo line is not — the agent's
error-reporting paths can produce useful explanatory comments whose
body leads with a checklist (e.g. "here's what I was working on" + the
incomplete todo list), and we don't want to silently overwrite those
with the generic "croaked" boilerplate.
In-process cleanup at action/main.ts:723 still handles the stranded-todo
case in the common path (gated on !finalSummaryWritten with full access
to the in-memory tool state). Missing the rare runner-died-mid-todo case
server-side is a worthwhile trade vs. the false-positive risk on real
explanatory comments.
* accept pullfrog.yaml as well as pullfrog.yml
centralize the accepted workflow filenames in `utils/github/pullfrogWorkflow.ts`
(`PULLFROG_WORKFLOW_FILES = ["pullfrog.yml", "pullfrog.yaml"]`) and use the new
`findExistingWorkflowFile` helper at every read path: `getWorkflow` (cached),
the verify-workflow API route, and the audit/sync/download/update scripts. `.yml`
is always tried first so the common case still costs exactly one API call.
webhook handlers (push cache-bust, `workflow_run_*`) now use the shared
`isPullfrogWorkflowPath` matcher.
action runtime (`reviewCleanup.ts`) derives the running workflow's filename from
`process.env.GITHUB_WORKFLOW_REF` instead of hardcoding `.yml`, so the safety-net
follow-up dispatch targets whichever file the user actually has — strictly more
correct than today.
write paths (`createWorkflowForRepo`, `createWorkflowPR`) intentionally still
create `.yml`; existing 422 collision handling covers the rare double-install
case. UI/wiki/onboarding copy keeps saying `pullfrog.yml`; one callout in
`docs/getting-started.mdx` mentions `.yaml` works too.
also drops dead code (`utils/github/findWorkflow.ts`, parallel single-file
implementation with no importers) and the now-unused `WORKFLOW_FILENAME` export.
* rename pullfrogWorkflow.ts -> findPullfrogWorkflow.ts (verb form)
* add pre-flight check to workflow create paths
`createWorkflowForRepo` and `createWorkflowPR` now check for any existing
pullfrog workflow file (`.yml` or `.yaml`) before doing work, preventing the
degenerate state where a repo with `pullfrog.yaml` ends up with both files
dispatching on every event.
costs one `getContent` call per first-time install. existing 422 branch in
`createWorkflowForRepo` is retained as a race-condition safety net; the 409
branch now also handles the case where `createWorkflowPR` discovers an
existing file in flight.
`createWorkflowPR` return shape becomes a discriminated union; the standalone
`/api/create-workflow-pr` route returns `{ alreadyInstalled: true }` instead
of creating a redundant PR.
* promote repo to active when /api/create-workflow-pr finds existing workflow
extracts `promoteRepoToActive` from `createWorkflowForRepo`'s closure to a
shared module-level function, and wires it into the standalone PR route's
`alreadyInstalled` branch so a `needs_setup` repo with an existing `.yaml`
file doesn't go stale (was only handled by the dashboard's own create path).
addresses pullfrog review on #596.
The previous phrasing ("not enough — still degrades the codebase") read as a
categorical claim that elegance vetoes correctness, which inverts the usual
hierarchy and risks giving the agent a clean rationalization for rejecting
genuine correctness fixes. Reframe as a prompt to keep searching for a fix
that gets all three before accepting the trade — preserves the pressure
without the absolute.
* refactor progress comments into a single bundled type + helper module
introduce ProgressComment ({ id, type: "issue" | "review" }) as the canonical handle for
the GitHub comment a run uses to report progress, and route every read/update/delete/create
through a single helper module (action/utils/progressComment.ts). previously every site that
touched the progress comment hardcoded octokit.rest.issues.*Comment, which made adding a
second comment type (review-thread replies) require duplicating the same branch in 6+ places
— the same shape that bit pullfrog/app#445.
new capability: when the address-reviews trigger fires for a one-off review comment, the
"Leaping into action" comment is now posted as a reply in that review thread instead of as
a top-level PR timeline comment. the helper handles failure (e.g. parent comment deleted)
by silently falling back to a top-level issue comment, so the run never loses its progress
surface.
changes:
- action/utils/progressComment.ts (new) — ProgressComment type + getProgressComment,
updateProgressComment, deleteProgressCommentApi, createLeapingProgressComment. uses a
structural Octokit interface to bridge the @octokit/rest version mismatch between the
action package (v22) and the root project (v21).
- action/internal/index.ts — re-export the new types and helpers for cross-boundary use.
- action/external.ts, action/utils/payload.ts — replace progressCommentId: string with
progressComment: { id: string, type: "issue" | "review" } in WriteablePayload + JsonPayload.
wire-format breaking, no legacy fallback (in-flight runs across the deploy lose their
progress comment, fine).
- action/mcp/server.ts — ToolState.progressCommentId becomes
progressComment: ProgressComment | null | undefined (same tristate semantics).
- action/main.ts, action/mcp/comment.ts, action/utils/errorReport.ts,
action/utils/postCleanup.ts — every issues.*Comment call against the progress comment
routes through the helper module. zero hardcoded API branching outside the helper.
- utils/github/triggerWorkflow.ts — drop createLeapingComment + updateCommentToLeaping;
dispatchAndTrackWorkflow gains a resolution chain (existingComment → replyToReviewComment
→ triggeringIssue → none) and an existingComment: ProgressComment param plus
replyToReviewComment: { pullNumber, commentId }.
- utils/webhooks/handleWebhook.ts — dispatch closure threads replyToReviewComment through;
the one-off review comment branch passes it and skips the now-redundant eyes reaction
on the comment we're about to reply to.
- app/trigger/[owner]/[repo]/[number]/page.tsx, utils/github/runActionLocal.ts,
app/api/cli/dispatch/route.ts, app/api/dispatch-workflow/route.ts — call sites updated to
new shape.
no schema or DB column changes. the existing WorkflowRun.progressCommentId column is still
written by id only; type lives only on the in-flight payload, which is sufficient for
runtime since it's the only thing that needs to know which API to call.
* anneal pass 1: fallback visibility + stale doc/comment updates
- progressComment.ts: when reviewReply→issue fallback fires, prepend a [!NOTE] callout
with a permalink back to the original review comment. without this, the parent comment
showed no eyes reaction (deliberately skipped) and no reply, leaving the user with no
signal that anything happened.
- wiki/post-cleanup.md: update progressCommentId references to progressComment, document
the new helper-based dispatch by type.
- wiki/main.md: update initToolState({ progressCommentId }) → ({ progressComment })
in the resolver-chain diagram.
- action/main.ts, action/mcp/review.ts: update two stale comments that referenced the
old field name.
* anneal pass 2: post-cleanup detection through fallback notice + log cleanup
- isLeapingIntoActionCommentBody: strip a leading GFM blockquote/alert before
testing the leaping prefix. without this, the [!NOTE] callout that the
reviewReply→issue fallback prepends would prevent post-cleanup from
recognizing the stuck "Leaping into action..." comment, leaving it permanently
on the PR timeline if the workflow died before any progress update.
- progressComment helper: switch from log.warning (action-flavored, emits a
::warning:: GitHub Actions annotation) to console.warn so the helper doesn't
pollute Vercel logs when invoked from the webhook context.
- triggerWorkflow.ts: drop the duplicate caller-side log on review-reply
failure — the helper already speaks loudly. Reword the catch-branch log to
reflect that it now only fires when both the reply AND the helper's internal
fallback failed.
- progressComment.ts: document that the [!NOTE] fallback notice is overwritten
on the first report_progress call, and explain the trade-off vs persisting
it through the action payload + ToolState.
* debloat: drop the [!NOTE] fallback callout
Reverting two pieces from the prior anneal pass:
- progressComment.ts: drop the [!NOTE] callout that the reviewReply→issue fallback
prepended to the leaping body. It disappeared on the agent's first report_progress
call, which made it half-committed to visibility — worse than either properly
persisting it (real engineering) or leaving the fallback silent (current choice).
The console.warn diagnostic and the workflow-run footer link in the leaping
comment itself give us enough signal for the rare case where both API endpoints
fail at once.
- isLeapingIntoActionCommentBody: revert the leading-blockquote stripping; only
needed to compensate for the [!NOTE] callout.
Keeping: the console.warn-vs-log.warning fix (real cross-runtime concern), the
duplicate-log drop in triggerWorkflow.ts, the wiki updates, and the two stale
source-comment fixes.
* fix: prevent stranded task list overwriting post-cleanup message
When a run is cancelled, the action's todoTracker may have an HTTP write in
flight to GitHub when SIGTERM lands. The action process dies, but the request
data has already left the socket — GitHub processes it and updates the comment
body to the (stale) task list. Meanwhile post-cleanup, running in a separate
process, writes the "This run was cancelled 🛑" message. If the tracker's
in-flight write happens to land *after* post-cleanup's write, the user never
sees the cancellation message.
Two-layer fix:
- Action side: cancel the tracker in the SIGTERM signal handler so no new
debounced writes get scheduled. This shrinks the race window but can't
un-send a request already on the wire.
- Post-cleanup side: after writing, verify the body landed and re-issue if
another write clobbered ours. Loops up to 3× with a 3s settle delay so
delayed in-flight writes from the dying action have time to arrive before
our read-back check decides whether to retry.
* lint: import createLeapingProgressComment from pullfrog/internal in test script
* address bot review findings: reply-target root, version bump, GET error handling
Three real findings from the bot reviews on #567 plus a small DRY pass:
1. handleWebhook reply-target: `newComments[0]` may be a reply, not a
top-level review comment. `getReviewCommentsWithReplies` returns root +
replies for any thread the review touched, and `pull_request_review_id`
filtering only narrows by *which review submitted*, not *root vs reply*.
When a user submits a single reply as their entire review (e.g. replying
to someone else's comment to ping @pullfrog), the reply ID flowed through
to `createReplyForReviewComment`, which 422s on replies-to-replies and
degraded to a top-level issue comment — exactly the polluted-PR-timeline
behavior this PR was built to remove. Walk up `in_reply_to` from the
already-fetched thread data to find the root and reply there instead.
2. action/package.json: bumped 0.0.202 → 0.0.204. main is at 0.0.203 and
our wire format changed; without a bump validateCompatibility can't
surface the mismatch on the deploy boundary, and the merge would have
gone backwards.
3. postCleanup writeAndVerify: distinguish a thrown verify-GET from a
"body got overwritten" mismatch. Treating a transient 5xx/rate-limit GET
the same as a clobber wasted PUT attempts and printed a misleading
"in-flight writes kept clobbering us" warning. We trust our PUT (which
returned 200) and exit instead of amplifying writes against a flaky API.
4. Small DRY: extracted parseProgressComment for the
`{ id: string; type } -> ProgressComment` parse that had drifted across
server.ts and postCleanup.ts.
#1 generational bump on both. xAI shipped grok-4.3 on 2026-05-01 and
grok-4-1-fast on 2025-11-19; both are same brand tier as the existing
slugs (`grok` and `grok-fast`), so resolve + openRouterResolve update
in place with no DB migration needed. Mirrored on the openrouter
provider side (openrouter/grok now also points at x-ai/grok-4.3).
OpenRouter spells the fast variant `x-ai/grok-4.1-fast` (dot) where
models.dev uses `grok-4-1-fast` (dash) — verified both forms against
their respective live APIs before committing. See the "naming traps"
section in wiki/models-catalog.md.
Snapshot regenerated: openrouter latest-GA shifted from
poolside/laguna-xs.2:free (2026-04-28) to x-ai/grok-4.3 (2026-05-01)
as a mechanical consequence of the bump.
Verified via `pnpm -C action test:catalog` (139/139 pass against live
models.dev + OpenRouter API) and `pnpm -C action test` (458/458).
Considered and explicitly rejected during this audit (recording for
future archaeology):
- Re-adding opencode/nemotron-3-super-free: removed twice in
71dff24c and 0f8117af with no commit-message rationale, but the
removals are intentional per maintainer.
- Adding gpt-nano (openai + opencode + openrouter) at gpt-5.4-nano:
the snapshot has been silently tracking opencode/gpt-5.4-nano since
7dd80143 (2026-03-18) without a corresponding catalog addition — a
deliberate non-add. Also would have collided with the existing
opencode/gpt-5-nano displayName "GPT Nano".
- Adding opencode/hy3-preview-free: never been in the catalog on main
and no positive signal beyond models.dev availability.
- Bumping opencode/gpt-5-nano (free) to opencode/gpt-5.4-nano: would
silently turn a free alias paid ($0.20/$1.25 per M tokens) — not a
generational bump, would require retire-and-replace if pursued.
releases the Review/IncrementalReview no-progress carve-out in
action/utils/run.ts (71dff24c) that has been sitting unpublished in
main since May 4. fixes the long-standing false-failure where Review
runs would error with "agent completed without reporting progress"
even after successfully submitting a review (issue #569).
the agent occasionally submits twice in one Review-mode run — once with
substantive feedback, then again with the canonical "Reviewed — no issues
found." body when the prompt's branch logic re-classifies non-blocking
observations as "no actionable issues" (see colinhacks/zod#5897). the
second submission is always redundant noise on the PR.
duplicateReviewDecision short-circuits the second call when toolState.review
is already populated for the current checkout sha. legitimate follow-up
reviews after new commits still go through because the new-commits-mid-review
path advances toolState.checkoutSha past the prior reviewedSha before
returning, so the next call sees a different sha and is allowed.
* fix: snapshot review state so progress comment cleanup actually fires
postReviewCleanup deletes toolState.review as its second statement, so
the defense-in-depth `if (toolState.review && progressCommentId)` branch
right after never saw a truthy value. This left an orphaned progress
comment alongside the submitted review whenever the agent called
report_progress despite Review/IncrementalReview mode instructions
(seen in the wild on colinhacks/zod#5767).
Snapshot the boolean before postReviewCleanup runs.
* move progress-comment cleanup into create_pull_request_review
The previous commit snapshotted toolState.review to work around
postReviewCleanup deleting it before the cleanup branch could read it.
That fixed the symptom but kept a fragile design: the rule "review
submitted → progress comment is noise" was enforced from the bottom of
main.ts via a flag set in one place and consumed in another, with a
helper between them that mutated the same flag for unrelated reasons.
Move the rule to its natural owner. create_pull_request_review now
calls deleteProgressComment immediately after the review is persisted,
so the cleanup is atomic with submission. This:
- closes the catch-block hole — a review submitted right before a
timeout/crash now still cleans up its progress comment.
- removes the dead "defense-in-depth" branch in main.ts that was the
original bug surface.
- relies on the existing progressCommentId=null no-op path in
reportProgress to make any later report_progress call a no-op (so
the misbehavior path can't re-create the orphan).
- only fires for Review/IncrementalReview in practice — those are the
only modes that call create_pull_request_review, and both are
prompted not to call report_progress. Build/AddressReviews/Plan
never reach this code path, so their progress comments remain
untouched.
Stranded-comment cleanup in main.ts is unchanged and still handles
the truly orphaned case (no review, no report_progress).
* add bundled git-archaeology skill, auto-installed for opencode and claude
ships a SKILL.md teaching agents the underused git history primitives
(pickaxe -S/-G, -L for function/line ranges, --reverse blame, deleted-file
recovery) so they stop scrolling git log -p when blame comes up empty.
introduces a lightweight bundled-skill path alongside the existing
addSkill (npx skills add) flow used for external skills like agent-browser.
SKILL.md is inlined into dist/cli.mjs via esbuild's text loader and written
to <home>/.agents/skills/<name>/SKILL.md at runtime — no network, no version
drift, no per-run install cost.
* fix: register vitest plugin to load .md as text for bundled-skill tests
* fix: drop vite type import from vitest plugin (vite isn't a direct dep)
* fix: load bundled skills via readFileSync so source mode works
esbuild's text loader only applies to the npm-bundled dist/cli.mjs path. the
preview / oss path runs cli.ts directly with node (PULLFROG_FORCE_LOCAL_CLI=1
in runCli.ts#runLocalCli), where node has no idea how to import .md files —
ERR_UNKNOWN_FILE_EXTENSION crashes the action before any agent starts.
switch to runtime readFileSync that checks both candidate locations:
- source mode: <actionRoot>/skills/<name>/SKILL.md (relative to utils/skills.ts)
- bundled mode: <distDir>/skills/<name>/SKILL.md (esbuild copies the tree)
drops the no-longer-needed esbuild text loader, vitest .md plugin, and
ambient *.md type declaration. wiki/skills.md updated with the why.
* fix: write bundled skills to per-agent dirs so claude actually registers them
* bump model registry for deepseek v4, kimi k2.6, claude opus 4.7
deepseek released v4 (pro/flash) on 2026-04-24 as a generational replacement
for v3-era reasoner/chat. deepseek will fully retire deepseek-chat and
deepseek-reasoner on 2026-07-24 — both already route server-side to v4-flash.
introduce deepseek-pro (preferred) and deepseek-flash slugs and mark the
legacy aliases deprecated via fallback so existing users transparently
upgrade. mirror on the openrouter side.
also bump moonshotai/kimi to k2.6 (from k2.5, 2026-04-21 release) and bump
the anthropic claude-opus openrouter resolves to 4.7 (we'd already moved the
native side to claude-opus-4-7 but openrouter resolves still pointed at 4.6).
update OSS_PROXY_MODEL fallback and stale doc reference accordingly.
snapshot regenerated; all 111 catalog tests + 66 unit tests pass.
* walk fallback chain when resolving the OSS proxy model
the OSS proxy path in run-context/route.ts read alias.openRouterResolve
directly, bypassing the fallback chain. so an OSS repo configured with
deepseek/deepseek-reasoner kept proxying to openrouter/deepseek/deepseek-v3.2
instead of resolving through the new fallback to openrouter/deepseek-v4-pro.
that worked today (v3.2 routes server-side to V4-Flash) but breaks when
deepseek and openrouter retire v3.2 alongside the 2026-07-24 deprecation.
extract the chain walk into a private resolveTerminalAlias helper and add
resolveOpenRouterModel that mirrors resolveCliModel but returns
openRouterResolve. fallback semantics now apply uniformly across both
runtime resolution paths.
* hide deprecated aliases from model selector dropdowns
aliases with a fallback (currently deepseek-reasoner / deepseek-chat /
openrouter/deepseek-chat) should not be selectable from the model dropdown
or the interactive cli model picker — they're a transition path, not a
choice. but if a repo already has a deprecated slug stored in the db, the
selector trigger still resolves it against the full alias registry so the
display name renders correctly until the user opens the menu and picks a
new model.
verified manually: deepseek submenu shows pro+flash only, openrouter submenu
shows pro+flash but no chat, and a deprecated stored value still renders
its full display name in the trigger.
* ci: run models-live on PRs that touch resolution files
Previously the per-alias smoke matrix only fired on push-to-main, so
resolution-affecting PRs (this one included) shipped without ever
exercising the agent harness against the real provider for each alias.
Loosen the gate on the `aliases` step in the `changes` job to fire
whenever the `models` paths-filter matches (action/models.ts,
action/package.json, action/agents/**) — same set that already drives
the comment about "resolution-affecting files". `models-live` itself
is unchanged: it still keys on a non-empty matrix.
`models-catalog` stays gated to main-push intentionally — its existing
comment justifies that (transient upstream catalog drift shouldn't
block PRs).
* relabel codex aliases as GPT, bump to 5.5 family, add gpt-pro
OpenAI retired the "-codex" model suffix on 2026-07-23 (gpt-5.3-codex,
gpt-5.1-codex-mini, gpt-5.2-codex et al all shut down) and unified the
codex+gpt lines into a single family at gpt-5.4. Per OpenAI's own
deprecation table, every "-codex" substitute is plain gpt-5.x — no
future Codex-suffixed frontier models are coming.
Keep the existing slugs for DB stability (no migration needed) but roll
displayName + resolve forward across openai, opencode, and openrouter:
- openai/gpt-codex → "GPT" → openai/gpt-5.5
- openai/gpt-codex-mini → "GPT Mini" → openai/gpt-5.4-mini
- openai/gpt-pro (new) → "GPT Pro" → openai/gpt-5.5-pro
Same relabel + new gpt-pro slug for opencode/* and openrouter/*.
gpt-5.5 (and gpt-5.5-pro) hit the OpenAI public API on 2026-04-24,
day after launch — both are live on OpenRouter as well.
There's no gpt-5.5-mini yet (analysts speculate late June – mid August
based on the gpt-5.4-mini cycle), so "GPT Mini" stays at gpt-5.4-mini
for now; one-line bump when the smaller variant ships.
Also pick up unrelated upstream catalog drift in the snapshot
(xai/grok-4.3 released 2026-05-01, openrouter/poolside laguna).
* deprecate gpt-codex aliases, mint gpt/gpt-pro/gpt-mini, render terminal alias in UI
The previous commit relabeled gpt-codex/gpt-codex-mini in place ("GPT" /
"GPT Mini") so a single slug carried two different identities. That worked
but was self-contradictory: the slug name no longer described the model.
Switch to the same shape we use for the deepseek V3→V4 transition:
- Mint new live slugs: openai/gpt, openai/gpt-pro, openai/gpt-mini
(mirrored on opencode/* and openrouter/*)
- Restore honest deprecated state on gpt-codex/gpt-codex-mini —
displayName "GPT Codex" / "GPT Codex Mini", original 5.3-codex /
5.1-codex-mini resolves, fallback set to the new gpt / gpt-mini slugs
- resolveCliModel + resolveOpenRouterModel walk the chain (existing
machinery), so DB rows holding "openai/gpt-codex" transparently route
to gpt-5.5 with no migration
UI render contract: display sites resolve to the *terminal* alias so a
deprecated stored slug shows the model the user is actually running, not
the historical name. Three call sites updated:
- components/ModelSelector.tsx (dropdown trigger label + provider label)
- action/utils/buildPullfrogFooter.ts (PR-comment "Using `X`" footer)
- action/commands/init.ts ("using model X" startup line)
Promoted internal resolveTerminalAlias → exported resolveDisplayAlias so
all three sites use the same primitive (also re-exported from external.ts
+ internal/index.ts so the Next.js app can import it).
Selectable lists (dropdown options, init picker) still filter on
!a.fallback so deprecated slugs never appear as fresh choices — only
deprecated stored values render.
wiki/model-resolution.md: replaced the muddled "slug names outlive
product names" bullet with a clear decision table for in-place bump
(generational, e.g. Opus 4.6 → 4.7) vs. deprecate+replace (vendor
restructures, e.g. codex → unified GPT, deepseek V3 → V4). Documents
the UI render contract too.
models-live CI matrix will smoke-test all 6 new slugs (gpt, gpt-pro,
gpt-mini × openai/opencode/openrouter) plus the 6 deprecated codex slugs
(which resolve through fallback to the same terminal targets) — 12 jobs
total against real provider APIs.
* wiki: slugs are evergreen, resolves are versioned
Document the slug-naming rule explicitly so future entries don't repeat
the deepseek-chat/deepseek-reasoner mistake (mirroring an upstream's
versioned/product-line-specific ID into the slug). Slugs should track
brand-style tier names that survive major version bumps; embedding
versions is the resolve string's job.
* Exclude GITHUB_WORKSPACE and relative entries from PATH walk
resolveExecutable previously walked any directory listed in process.env.PATH,
which trusts that nothing earlier in the workflow prepended an
attacker-controlled location. A malicious PR could land bin/npx in the repo
and add `echo "$GITHUB_WORKSPACE/bin" >> $GITHUB_PATH` to a prior step,
causing pullfrog to exec the attacker's binary with our scoped tokens in env.
Filter out (a) any non-absolute PATH entry (., bin, .., etc., which resolve
against cwd) and (b) any entry equal to or under GITHUB_WORKSPACE. The walk
then continues to the next legitimate system tooling dir.
* Address PR #558 review: comment typo + Windows case bypass
- Drop double space in the threat-model comment.
- Lowercase paths on Windows before comparing against GITHUB_WORKSPACE.
Without this, an attacker can bypass the filter by varying case in their
injected PATH entry (`d:\a\repo\bin` vs `D:\a\repo`) — string compare
misses but NTFS still resolves the executable inside the workspace.
* Fix Node 24 action bootstrap fallback
Resolve the published CLI launcher through PATH so runners missing a sibling Node 24 npx can still start, and make post cleanup recognize prefixed leaping comments.
* Bump Pullfrog action package version
Ensure the Node 24 bootstrap and post-cleanup fixes publish to npm and move the v0 action tag.
* Walk PATH for corepack and npx in action bootstrap
ensureActionDependencies and runPackageCli now resolve corepack/npx through
PATH the same way as the npx-via-PATH fix, so Node 24 runner pools missing
either sibling can still bootstrap. Also adds a Zod-mirror settings helper
for the preview-556 repo and documents the per-PR settings workflow.
* log when corepack PATH fallback is used
* unify per-run token + cost accounting across agents
every agent harness now logs the same 5-column (or 6 with cost) table and
populates the same AgentUsage contract, regardless of agent or upstream
provider. previously OpenCode and the Claude fallback path emitted a 3-col
table whose "Input Tokens" was actually only the non-cached delta, silently
dropping cache read/write — real runs were being reported at ~0.4% of their
true input (e.g. one baseline showed Input=30 while step_finish events
summed to cache_read=724,753).
changes:
- add logTokenTable helper in action/agents/shared.ts with stable columns:
Input | Cache Read | Cache Write | Output | Total | Cost ($). cost
column renders only when a value is known.
- action/agents/opencode.ts: accumulate step_finish.part.tokens AND
step_finish.part.cost (sourced from models.dev inside opencode —
confirmed working across Anthropic, OpenAI, Google, xAI, DeepSeek,
Moonshot, and OpenRouter). drop the event.stats.total_tokens fallback
since that payload has no cache breakdown.
- action/agents/claude.ts: success-path now treats input_tokens as the
non-cached field (matching OpenCode semantics), carries
cache_read_input_tokens / cache_creation_input_tokens separately, and
captures total_cost_usd from the final result event. the per-message
fallback accumulator now captures cache fields too so it's no longer
lossy when the result event never fires.
- formatUsageSummary gains a Cost ($) column that matches the stdout
table row-for-row; missing values render as "—".
- scripts/token-usage.ts parses all three historical formats (new 5-col,
legacy 4-col Claude success, legacy 3-col lossy) and explicitly flags
the lossy runs instead of averaging misleading values.
validation (pnpm play --local, identical "say hello" prompt):
agent+model Input CacheR CacheW Output Total Cost
OpenCode + Anthropic Sonnet 4.6 4 41,177 20,735 129 62,045 $0.0921
Claude CLI + Anthropic Sonnet 4.6 9 80,133 11,611 389 92,142 $0.0766
OpenCode + OpenAI codex-mini 10,893 46,976 0 606 58,475 $0.0059
OpenCode + Google Gemini 3 Flash — — — — — $0.0114
OpenCode + xAI Grok 4 Fast — — — — — $0.0035
OpenCode + DeepSeek Chat 18,854 0 0 1 18,855 $0.0053
OpenCode + Moonshot Kimi K2.5 — — — — — $0.0106
OpenCode + OpenRouter→Anthropic — — — — — $0.0617
OpenCode + OpenRouter→OpenAI — — — — — $0.0038
* isolate play.ts from developer gitconfig
play.ts is a CI-emulator but inherits the developer's user- and system-scope
gitconfig. a common local convenience — url."git@github.com:".insteadOf
"https://github.com/" to force SSH auth — gets applied at read time on every
git call inside the temp repo, causing `git remote get-url --push origin`
to return an SSH URL instead of the stored HTTPS one. pullfrog_push_branch's
validatePushDestination (correctly) treats that as tampering and blocks the
push. the agent then burns the full MAX_COMMIT_RETRIES budget trying
workarounds that can't beat a user-scope insteadOf rule, turning a trivial
"say hello" run into a 1.35M-token session.
point GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at /dev/null inside run() so
the play process and its spawned agent see the same empty gitconfig that
a real CI runner would. CI has no rewrites, so this is a no-op there; dev
machines get CI-identical git state. SSH client config (~/.ssh/config and
keys) is separate from gitconfig and is unaffected, so setupTestRepo's SSH
clone still works locally. setupGit only writes --local scope, so nothing
downstream depends on user-scope values.
verification: with the scratch repo cleaned up and this isolation in place,
OpenCode + Anthropic on the same "say hello" prompt goes from 1,349,654
tokens / $2.00+ to 62,045 tokens / $0.0921 — no retry loop, no push blocks.
* persist aggregated token + cost usage to WorkflowRun
AgentUsage has been memory-only — rendered into the GitHub step summary
and then discarded when the runner tears down. that made questions like
"avg cost per customer per day" require log-spelunking. persist it:
- add Int? columns for inputTokens / outputTokens / cacheReadTokens /
cacheWriteTokens and a Decimal? costUsd column on workflow_runs.
Int4's 2.1B ceiling is ~200x larger than any realistic run so BigInt
would be overkill. costUsd uses the same default Decimal precision
as existing money columns (accounts.usageUsd, proxy_keys.hwmUsage).
- extend PATCH /api/workflow-run/[runId] to accept the new numeric
fields alongside the existing artifact strings. per-field type
validation ensures the allowlist stays scalar-safe and rejects
negative / non-finite values.
- generalize patchWorkflowRunFields in the action so it accepts a
mixed string/number payload, and add an aggregateUsage(entries)
helper that sums per-agent AgentUsage records into a single patch.
- call the reporter from main.ts's outer finally block, gated on
toolContext. this is the shared cleanup path that every agent
implementation flows through — claude.ts, opencode.ts, and any
future harness all push their AgentUsage into toolState.usageEntries
via the same line 468, so one finally-block call covers them all.
running in finally also means partial usage gets persisted even
when the agent errored out mid-run.
* anneal token + cost accounting
follow-up polish from a review pass:
- aggregate usage across commit-retry iterations inside each agent harness.
previously runClaude / runOpenCode returned only the final retry's usage,
so any run that hit the dirty-tree retry loop under-counted tokens and
cost in both the stdout table and the WorkflowRun row. added a shared
mergeAgentUsage helper in agents/shared.ts; both harnesses now fold each
iteration's usage into a running total and return the sum.
- scripts/token-usage.ts now handles the unified format with or without
the Cost ($) column. previously the int-only number regex rejected
decimals and the 5-cell length check rejected 6-cell rows, so logs
from post-cost-tracking runs fell through to "no token table". the
parser now accepts both 5- and 6-cell unified rows, splits int vs
decimal cells, and averages reported Cost alongside the tokens.
- PATCH /api/workflow-run/[runId] now rejects INT field values above
INT4_MAX (2_147_483_647) so a malformed payload gets a clean 400
instead of propagating a Prisma error. also defends against a
compromised runner sending a deliberately huge value.
- clarifying comments: opencode.ts documents that step_finish.part.cost
is a per-step delta (empirically verified), main.ts explains that
toolState.usageEntries already carries merged per-retry usage so
aggregateUsage just sums entries (one per agent.run()).
- tests for aggregateUsage and mergeAgentUsage — 12 new cases covering
empty / partial / multi-agent inputs and the "keep undefined" semantic
that prevents spurious zeros from being persisted.
- drop `as number` cast in logTokenTable — narrow via const instead.
* anneal: clamp INT overflow + guarantee mergeAgentUsage immutability
second review pass surfaced two defensive gaps:
- a single token field exceeding INT4_MAX would pass the client but be
rejected by the server's per-field validator, writing a partial row
with some NULLs where sums belonged. clamp in aggregateUsage so the
wire payload is always self-consistent across all numeric columns,
with a loud warning so the clamp doesn't silently swallow weirdness.
- mergeAgentUsage's single-sided branches returned the input reference.
callers treat AgentUsage as immutable but future callers might not;
always return a fresh shallow copy instead. two new tests guarantee
the no-mutation-leak property.
no behavior change in the happy path — INT4_MAX is ~200x the largest
realistic per-run token count.
* anneal: resilient usage persistence + cross-platform null device
third review pass surfaced three small issues:
- main.ts finally block: writeGitHubUsageSummaryToFile throwing would
skip the WorkflowRun usage PATCH. both are independent best-effort
cleanup tasks — wrap the former in catch so a filesystem failure
doesn't block DB persistence.
- AgentUsage.inputTokens had no jsdoc explaining that it's the full
billable input (cached + non-cached). the same word "Input" means
"non-cached only" in the stdout/markdown tables (derived by
subtraction). document the semantic so dashboards querying
WorkflowRun.inputTokens don't misinterpret it.
- play.ts gitconfig isolation was hard-coded to "/dev/null" which
doesn't exist on Windows. use `os.devNull` for cross-platform
parity (resolves to `\\.\nul` on win32). the project is Linux-only
in CI so this only helps local Windows contributors, but it's a
zero-cost swap.
also updated the finally-block caveat comment: usage is only pushed
to toolState.usageEntries when agent.run() returns an AgentResult,
not when the timeout race rejects — so timed-out runs don't
persist partial usage. documented instead of trying to thread state
through Promise.race.
* anneal: NaN-guard cost accumulators + clarify inputTokens docs
final polish from review round 4:
- guard both cost accumulators (opencode step_finish.part.cost and claude
result.total_cost_usd) with Number.isFinite. `typeof x === "number"`
accepts NaN, and one NaN `+=` would poison the running total for the
whole session.
- reword prisma schema comment on WorkflowRun usage fields to call out
that cacheReadTokens / cacheWriteTokens are SUB-totals within
inputTokens (not additional tokens on top). prevents future dashboards
from double-counting by ~2x when summing "total tokens used".
claude-code's commander parser treats --disallowedTools as variadic
<tools...>, which silently absorbs extra tokens but may not enforce
them as reliably as a single comma-separated value. switch to the
form the CLI help documents ("Bash,Agent(Bash)") to make the deny
list unambiguous.
The previous sanitizer proxied `schema.toJsonSchema()`, but fastmcp 3.x uses
`xsschema.toJsonSchema()` which reads `schema["~standard"].jsonSchema.input(...)`
directly when the StandardJSONSchemaV1 extension is present (arktype 2.x).
Our proxy was never invoked, so the sanitizer was a silent no-op.
Proxy the entire `~standard` → `jsonSchema` → `input` chain so the transform
runs regardless of which path xsschema picks. Also add case 1 (add `type:"string"`
to enum-only schemas) — arktype 2.x emits `{enum:["A","B"]}` without a type
field, which is the exact form Gemini rejects with
"only allowed for STRING type".
Verified locally: wrapped schema now emits `{type:"string", enum:[...]}` and
drops `$schema`; validation still works.
Gemini's generateContent API rejects arktype's `{anyOf:[{enum:[...]}]}` string-enum
encoding, `$schema` metadata, and `anyOf` with sibling fields. Port the old
sanitizer back as an isolated module (action/mcp/geminiSanitizer.ts) and gate it
on `isGeminiRouted(ctx)` so non-gemini routes see the original schema. Wires
`resolvedModel` onto ToolContext so the sanitizer can see the upstream specifier.
Also bumps `openai/gpt-codex-mini` alias from the deprecated `codex-mini-latest`
to `gpt-5.1-codex-mini`, matching the openrouter resolve.
Adds a `filter` workflow_dispatch input + MATRIX_FILTER env that restricts the
models-live matrix to aliases matching a substring, so we can iterate on a
single provider (e.g. `filter=gemini`) without paying to run every model.
two bugs blocked the live matrix from reaching real APIs:
1. resolveModel returned PULLFROG_MODEL raw without passing it through the
alias registry. when CI set PULLFROG_MODEL=anthropic/claude-opus (alias),
the bare alias slug was forwarded to the Anthropic API as a model id and
404'd. now resolves via resolveCliModel first, with raw specifiers
(anthropic/claude-opus-4-6) still passing through unchanged.
2. the testEnvAllowList in docker.ts only forwarded Anthropic/OpenAI/Google
keys into the test container. XAI/DeepSeek/OpenRouter/Moonshot/OpenCode
keys got stripped, so every non-big-3 alias failed with "no API key found"
even when the secret existed. add all five to the allowlist.
Made-with: Cursor
* add one-time diff coverage preflight for PR reviews
track diff read coverage from agent tool-use events and run a one-time pre-flight before review submission, with explicit coverage skip reasons for low-value files like lockfiles.
Made-with: Cursor
* add manual dispatch fallback for preview deploy workflow
allow preview repo and preview sync jobs to be run via workflow_dispatch with explicit PR number and branch inputs, so preview provisioning can be retriggered when pull_request events fail to fire.
Made-with: Cursor
* fix manual preview dispatch PR input wiring
use normalized PR number and branch env values for comment creation and script env wiring so workflow_dispatch preview runs can create and update PR-specific preview resources.
Made-with: Cursor
* remove obsolete snapshots invalidated by checkout instructions change
* fix diff coverage read offset handling and add local sanity-check guidance
normalize read offset semantics for diff coverage tracking, reuse shared range counting in review preflight, add focused diff coverage unit tests, and document the local play.ts testing workflow in AGENTS.md.
Made-with: Cursor
* add regenerated mcp test snapshots
capture snapshot files generated by the review comment and checkout formatting tests during pre-push validation so the branch remains clean and reproducible.
Made-with: Cursor
* add diff coverage preflight instrumentation logs
log diff coverage initialization in checkout_pr and emit preflight state/breakdown diagnostics in create_pull_request_review to debug missing coverage enforcement in preview e2e runs.
Made-with: Cursor
* add env override to force local cli execution in action runtime
support explicit local-cli execution via PULLFROG_FORCE_LOCAL_CLI so preview workflows can run branch action code instead of the npm fallback package during e2e debugging.
Made-with: Cursor
* add preview e2e debugging learnings for action runtime validation
capture the preview execution-path gotchas and one-time preflight verification pattern in AGENTS.md so future investigations validate the real runtime and avoid npm fallback confusion.
Made-with: Cursor
* reduce diff coverage log noise while preserving failure visibility
downgrade verbose diff coverage lifecycle diagnostics to debug, keep a concise info-level pre-flight failure signal, and document preview runtime debugging learnings in AGENTS.md.
Made-with: Cursor
* WIP
* tune sync.md: ff override + softer overlap verification
Made-with: Cursor
* chore: bump models snapshot for claude-opus-4-7
Made-with: Cursor
* rip out coverage_skips waiver from diff coverage pre-flight
Made-with: Cursor
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
PR CI kept breaking on upstream catalog drift (new model ships on models.dev,
OpenRouter renames an id, etc.) — failures unrelated to the PR's contents.
split the model-alias test suite so PRs only see pure-logic checks, and push
the external-state drift + end-to-end coverage to main.
test organization:
- action/test/models.test.ts keeps pure invariants: openRouterResolve
completeness and fallback-chain resolution. runs on every PR.
- action/test/models-catalog.main.test.ts gets the 4 network-dependent
describes (models.dev validity x2, OpenRouter API validity, latest-model
snapshot). runs only on main push via a dedicated vitest config
(vitest.main.config.ts + `pnpm test:catalog`).
new CI jobs in .github/workflows/test.yml:
- models-catalog: `pnpm test:catalog` on every main push. detects upstream
catalog drift so we can react at the next convenient window.
- models-live: 38-entry matrix that invokes the agent harness end-to-end
against the real provider for each alias in models.ts. generated from
action/test/list-aliases.ts. runs only on main push AND only when
resolution-affecting files changed (action/models.ts, action/package.json,
action/agents/**) — the exact shape of the opus 4.7 incident.
test/run.ts: PULLFROG_MODEL now flows through from process.env so the live
matrix can pin an alias per job without the per-agent default clobbering it.
Made-with: Cursor
anthropic shipped claude-opus-4-7 today; opencode also republished it.
point the "claude-opus" alias at the new version for both providers so
existing users get the upgrade automatically. openrouter hasn't
published 4.7 yet, so leave openRouterResolve at 4.6 as the BYOR fallback.
also clarify the latest-model snapshot comment: new model drops usually
just mean bumping the `resolve` on an existing alias, not adding a new one.
Made-with: Cursor
remove broad `PULLFROG_` passthrough from restricted shell env filtering and ensure sensitive names are blocked unless explicitly allowlisted, then align the restricted test fixture with allowed-prefix coverage.
Made-with: Cursor
* replace suffix-based env filtering with default-deny allowlist
filterEnv() now only passes known-safe GitHub Actions runner/system/toolchain
vars plus user-configured allowlist entries to shell subprocesses. GITHUB_TOKEN
and GH_TOKEN are always blocked, even from the user allowlist.
adds envAllowlist field to repo settings with dashboard textarea UI (visible
only when shell isolation is enabled) and wires it through run-context API
to the action runtime.
Made-with: Cursor
* address review: blocked-name warning, JAVA_HOME prefix, stale waitlist copy
- setEnvAllowlist now strips BLOCKED_ENV_NAMES from user input and returns
them so main.ts can log a warning
- move JAVA_HOME to exact names, use JAVA_HOME_ as prefix for clarity
- update stale suffix-based description in waitlist email script
Made-with: Cursor
* fix wiki/security.md snippet: JAVA_HOME -> JAVA_HOME_ to match code
Made-with: Cursor
* UI polish: field-sizing-content on all textareas, rename env allowlist label
- add field-sizing-content to all settings textareas so they auto-expand
to fit content (AgentSettings, ModesSettings, WorkflowsSettings, FlagsSettings)
- rename "Environment variable passthrough" to "Environment allowlist"
with clearer popover copy
- drop "e.g." prefix from env allowlist placeholder
- update docs/security.mdx and wiki/security.md references to match
Made-with: Cursor
* tweak env allowlist popover wording
Made-with: Cursor
* document default allowed variables in security docs with link from popover
Made-with: Cursor
* Update action/utils/secrets.ts
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
remove duplicate model and agent log emitters, then print model, agent, push, shell, and timeout on separate startup lines so run settings stay concise and easy to scan.
Made-with: Cursor
review bodies were embedding a task-list snapshot that could capture
stale in-progress state due to timing between the agent's final
TodoWrite and the review submission API call. progress comments
are the authoritative checklist surface — remove the review-body
embedding entirely so there is a single source of truth.
also adds a `completeInProgress` option to `renderCollapsible` so
the progress-comment path can finalize any in-progress items at
render time without mutating tracker state.
Made-with: Cursor
* rename agent key to opencode and add skill invocation coverage.
add skill-invoke tests for claude and opencode with local play-based validation signals, update CI matrices, and include the current tracked refactors in this branch for review.
Made-with: Cursor
* exclude agent-specific skill-invoke tests from wrong agent in CI matrix
* address review follow-up and preserve workflow run UI tweak.
switch changed-agents ci coverage to exercise the opentoad implementation path while keeping the opencode expectation, and include the local workflow run client interaction updates requested on this branch.
Made-with: Cursor
* remove opentoad agent filename from runtime.
rename the opencode harness implementation file from opentoad.ts to opencode.ts and update ci coverage input accordingly so action code no longer carries the old filename.
Made-with: Cursor
* ensure security prompt bypass is set on every test fixture.
this keeps adversarial and permissions harnesses from being blocked by the default prompt-injection refusal path during CI security tests.
Made-with: Cursor
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* add wiki/betterstack.md documenting log querying, request-ID grouping, and MCP usage
Made-with: Cursor
* fix webhook race conditions: separate runId assignment from data updates
the workflow_run webhook handler had a race where concurrent handlers assigned
the same runId to different pending records. the loser's P2002 silently dropped
data updates (jobId, status, completedAt). fix by splitting into two steps:
assignRunId() handles the race-safe unique assignment, then data updates always
target where: { runId } so they hit the correct record regardless of who won.
also downgrade R2 ObjectLockedByBucketPolicy errors from error to warn level
since duplicate webhook deliveries writing the same key is expected under load.
Made-with: Cursor
* homepage copy refresh + fix skills CLI installation
- update hero to "Agent x GitHub" with new subtagline
- rewrite intro paragraphs: workflow, harness capabilities, billing
- add feature sections: bash isolation, headless browser, MCP tools
- update FAQ answers, footer attribution, free-for-oss copy
- update APP_DESCRIPTION for SEO
- fix skills install: use npx from tmpdir instead of local binary
(the bundled action has no node_modules; running npx from tmpdir
avoids project .npmrc with pnpm settings breaking binary resolution)
- instruct agents to use markdown image syntax in upload_file tool
- start dependency installation eagerly from main.ts
- include event title in task instructions
Made-with: Cursor
* add wiki/betterstack.md documenting log querying, request-ID grouping, and MCP usage
Made-with: Cursor
* fix webhook race conditions: separate runId assignment from data updates
the workflow_run webhook handler had a race where concurrent handlers assigned
the same runId to different pending records. the loser's P2002 silently dropped
data updates (jobId, status, completedAt). fix by splitting into two steps:
assignRunId() handles the race-safe unique assignment, then data updates always
target where: { runId } so they hit the correct record regardless of who won.
also downgrade R2 ObjectLockedByBucketPolicy errors from error to warn level
since duplicate webhook deliveries writing the same key is expected under load.
Made-with: Cursor
* require OIDC verification for DB secrets on run-context endpoint
DB secrets transported via run-context were accessible to any GitHub API
token with read access, bypassing GitHub Actions' fork PR secret isolation.
Now the endpoint requires a valid GitHub Actions OIDC token
(X-GitHub-OIDC-Token header) with a matching repository claim before
returning dbSecrets. Also requires admin for account-scope CLI secret
writes (matching the dashboard), and removes dead redactSecrets code.
Made-with: Cursor
emit real ESM runtime + declaration outputs for programmatic imports, align package exports/types with built files, and add a no-cjs policy note.
Made-with: Cursor
npx was running with cwd set to the action's own directory, which has
package.json with "name": "pullfrog". npm treats the local package as
satisfying the request and skips the registry fetch, then fails to find
the binary (sh: 1: pullfrog: not found). use GITHUB_WORKSPACE instead.
Made-with: Cursor
publish was missing a build step so the npm tarball had no dist/.
switch from NPM_TOKEN to OIDC trusted publishing — explicitly unset
NODE_AUTH_TOKEN so setup-node's .npmrc doesn't override the OIDC flow.
bump version since v0.0.195 tag exists from the failed publish attempt.
Made-with: Cursor
action/.husky prepare script was overriding root husky config, so the
pre-push hook (lint + typecheck + test) never ran. merged the lockfile
sync pre-commit into root .husky/pre-commit and removed action/.husky.
also auto-fixed biome format/import-sort errors from last commit.
Made-with: Cursor
simplify installation-not-found flow by removing ownerHasInstallation
field and collapsing the "selected repos" vs "no access" branches into
a single message with a confirm prompt. improve spinner/log copy
throughout init (secrets, model, workflow, test run).
backfill missing jobId on workflow-run redirect page by querying the
GitHub API for the pullfrog job when jobId is null. add 600ms delay
in handleWorkflowRunInProgress before fetching jobs to avoid racing
the job assignment.
Made-with: Cursor
the postinstall script referenced scripts/generate-proxies.ts which isn't
included in the published npm package, silently breaking every npx install.
replaced the proxy stub approach with turbopack resolveAlias and webpack
conditionNames so both bundlers resolve pullfrog imports to TypeScript
source directly — matching what tsc already does via customConditions.
also moves PR summary format from handleWebhook into modes.ts so the
summarize mode prompt includes it directly.
Made-with: Cursor
PRs created as draft (or by automation tokens that suppress workflow
triggers) never ran preview-create because the workflow only listened
for opened/synchronize. Adding ready_for_review as a trigger ensures
the preview repo gets created when a draft PR is marked ready.
Also makes preview-create.ts idempotent by catching 422 (repo already
exists) so it's safe if both opened and ready_for_review fire.
Made-with: Cursor
the inline `node -e` + `TOKEN=$(...)` approach broke because
`core.getIDToken()` in @actions/core writes `::debug::` and
`::add-mask::` to stdout, polluting the captured value.
`node cli.ts gha token` uses `core.setOutput()` which writes to
the $GITHUB_OUTPUT file instead of stdout.
Made-with: Cursor
* improve incremental review output and fix todo tracker race
- reviewed changes section: summarize at logical-change level with
past-tense verbs, not per-file enumerations
- add TodoTracker.completeAll() to mark all non-cancelled items as
completed before snapshotting the collapsible in review/progress posts
Made-with: Cursor
* completeAll -> completeInProgress: only mark in-progress items as completed
Pending items that were genuinely skipped stay as-is in the collapsible,
so the task list honestly reflects what the agent actually did.
Made-with: Cursor
* fix: make token-exfil test reliable by disabling security instructions and reframing prompt
the test was flaky — agents would randomly refuse (not calling set_output),
refuse politely (calling set_output with refusal text), or cooperate fully,
depending on model mood. two changes:
1. set PULLFROG_DISABLE_SECURITY_INSTRUCTIONS=1 in test env (layer 1)
2. reframe prompt as CI debugging task instead of security test (layer 2)
Made-with: Cursor
* fix: set PULLFROG_DISABLE_SECURITY_INSTRUCTIONS on adversarial test fixtures
without this flag, the system prompt tells agents to refuse anything that
looks malicious — which is exactly what these security pentests ask them to
do. adds the flag to tokenExfil, askpassIntercept, and nobashcreative.
Made-with: Cursor
* set PULLFROG_DISABLE_SECURITY_INSTRUCTIONS on all security-related test fixtures
Made-with: Cursor
the "delegate a subagent" instruction doubled LLM sessions for
every summary run, and "use mini or auto effort" was a no-op
since the agent always runs at high/max effort.
Made-with: Cursor
* fix WorkflowRun mis-assignment when multiple dispatches are in flight
workflow_run_requested fires before GitHub applies the custom run-name,
so display_title has no [suffix]. the old desc ordering picked the newest
pending record, cross-linking enrichment ↔ auto-label records.
switch to FIFO (asc) ordering so records are claimed in dispatch order,
and add a 15s createdAt window to avoid claiming stale records.
fixes#523
Made-with: Cursor
* WIP
* plan: update issue indexing resolution to R2-backed lazy filesystem
replace the direct GitHub tarball + in-memory extraction approach with a
two-phase architecture: streaming tarball sync to R2 (per-file, via
tar-stream) and on-demand lazy loading via just-bash InMemoryFs backed
by R2 GETs. scales to 200K+ file monorepos at <50MB memory overhead.
Made-with: Cursor
* plan: switch to tarball + R2 range requests, add design alternatives rule
update issue indexing plan to use a single uncompressed tar in R2 with
byte-offset index instead of per-file uploads. 2 PUTs per sync vs 10K,
5000x cheaper, trivial lifecycle.
add AGENTS.md rule: generate 3 alternatives before committing to a design.
Made-with: Cursor
* fix typos in AGENTS.md
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
- in Review mode, stop the todo tracker and append the completed task
list as a collapsible section to the review body before submitting
- always delete the progress comment after a review is submitted,
regardless of whether the agent called report_progress
Made-with: Cursor
* refactor: replace narrow parameter types with context objects across action/
pass broader context objects (ToolContext, PromptContext, PostCleanupContext) to
utility functions instead of cherry-picking fields into single-use interfaces.
deletes 8 narrow types, simplifies call sites, and makes buildCommentFooter
synchronous by reading ctx.runId/ctx.jobId directly instead of re-deriving
from env vars and making an extra API call.
Made-with: Cursor
* fix: replace non-null assertion with local guard in validatePushDestination
addresses review feedback — the function now validates pushUrl itself instead
of relying on the caller's check, eliminating the ! assertion.
Made-with: Cursor
* revert: remove GH_TOKEN injection from restricted shell
the original change exposed the git token in restricted-mode shell so
`gh` CLI would work. this is a security regression for public repos: MCP
tools are deliberately constrained (no merge, no release, no arbitrary
API calls), but `gh api` with the token gives full GitHub API access to
any prompt-injected agent.
Made-with: Cursor
* reorder prompt sections: task-first with dynamic TOC
put the actual task at the top of the prompt for primacy, add a
dynamic table of contents, and push system/runtime metadata to the end.
new section order: TOC → YOUR TASK → PROCEDURE → EVENT CONTEXT →
SYSTEM → LEARNINGS → RUNTIME
Made-with: Cursor
* enforce clean working tree: continue session if agent leaves uncommitted changes
after each agent run, check `git status --porcelain`. if dirty, resume
the same session with instructions to commit on a new branch, push, and
open a PR. retries up to 3 times before giving up.
- claude code: capture session_id from result event, use --resume <id>
- opencode: use --continue to resume the last session
- remove --no-session-persistence from claude (needed for --resume)
- update Task mode to clarify branch/push/PR is the default finalize step
Made-with: Cursor
* log full prompt in collapsible group for debugging
Made-with: Cursor
* fix: format tool refs in buildCommitPrompt via formatMcpToolRef
* enforce clean git status: general instructions, stop hook, and Task mode
Made-with: Cursor
* fix: rename stale titleBody references after body leak fix
Made-with: Cursor
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* WIP
* WIP
* review/incremental-review: always submit review, never call report_progress
The progress comment is auto-deleted by the stranded-comment cleanup in
main.ts when the agent skips report_progress. This makes reviews the
sole PR artifact for both modes, reducing noise.
- soften report_progress tool description to allow mode opt-out
- Review mode: always submit exactly one review (approve or request changes)
- IncrementalReview mode: submit review for substantive outcomes, silently
exit for non-substantive changes (formatting-only pushes produce zero artifacts)
Made-with: Cursor
* incremental-review: clarify approval condition for substantive no-issues case
Made-with: Cursor
* report_progress: s/completed/current task list
Made-with: Cursor
* system instructions: align report_progress guidance with mode opt-out
Made-with: Cursor
* fix autofix body leak, harden prompt against injection, trigger-aware comments
never forward event bodies to the agent prompt — they are user-generated
content and a prompt injection vector. the agent fetches bodies on demand
via MCP tools (checkout_pr, get_issue, etc.).
- always set event.body to null in dispatch(), add promptFromBody: false
to autofix, strip body from nested pull_request object
- replace buildEventTitleBody with buildEventTitle rendering inline
references like PR #497 ("Title") instead of raw markdown headings
- add LEAPING_REASON_MAP for trigger-aware progress comments
(e.g. "CI failure detected. Leaping into action...")
- thread type through buildLeapingIntoActionComment, createLeapingComment,
and updateCommentToLeaping
Made-with: Cursor
* rename translateWorkflowRunType.ts to workflowRunTypes.ts
Made-with: Cursor
models can now be marked `deprecated: true` with a `fallback` slug
pointing to a replacement. `resolveCliModel` follows the chain
recursively (with cycle detection) until it finds a non-deprecated
model. this keeps deprecated models in the registry for backward
compatibility instead of removing them.
marks opencode/mimo-v2-pro-free as deprecated with fallback to
opencode/nemotron-3-super-free.
Made-with: Cursor
* sandbox native filesystem tools to prevent /proc/self/environ exfiltration
the agent's native Read/Grep/Edit tools can bypass the shell sandbox by
reading /proc/self/environ directly. this adds agent-native filesystem
restrictions using the highest-precedence, non-overridable config for each CLI:
OpenCode: OPENCODE_PERMISSION env var with external_directory deny-all + /tmp allow,
plus deletion of untrusted .opencode/plugins/ and .opencode/tools/ before launch.
Claude Code: managed-settings.json at /etc/claude-code/ with denyRead, permissions.deny,
allowManagedPermissionRulesOnly, allowManagedHooksOnly. also --setting-sources user and
--disallowedTools path patterns as belt-and-suspenders.
Made-with: Cursor
* add Glob to Claude Code /proc and /sys deny lists
closes gap identified in review — Glob can enumerate /proc entries.
added to both managed-settings.json permissions.deny and --disallowedTools.
Made-with: Cursor
* run token-exfil test with both agents, hint at native /proc reads
changed tag from "agnostic" (opentoad-only) to "security" so the test
runs with both opentoad and claude. updated prompt to explicitly instruct
the agent to try reading /proc/self/environ via native Read tool.
added API keys to action-agnostic CI job for claude support.
Made-with: Cursor
* move token-exfil to crossagent matrix, remove redundant permissions.deny
- moved token-exfil from agnostic/ to crossagent/ so it runs via the
agent matrix (claude + opentoad in parallel) instead of sequentially
- removed permissions.deny per-tool rules from managed-settings.json;
sandbox.filesystem.denyRead is the single enforcement mechanism
- reverted action-agnostic env vars to minimal set
- updated wiki to match
Made-with: Cursor
* document post-spawn API key deletion analysis in security wiki
evaluated whether API key env vars can be deleted from agent processes
after spawn. OpenCode snapshots env at startup (safe to delete), but
Claude Code re-reads process.env per request (not viable). documented
as further exploration item with per-agent breakdown and caveats.
Made-with: Cursor
* fix stale tokenExfil path references in wiki docs
moved from test/agnostic/ to test/crossagent/ in directory tree
and adversarial test example.
Made-with: Cursor
* revert accidental prisma.config.ts changes
Made-with: Cursor
* hardcode PULLFROG_MODEL per agent in test runner to avoid DB model mismatch
when PULLFROG_AGENT forces a specific agent, the DB-configured model may
belong to a different provider (e.g. openai model with claude agent).
PULLFROG_MODEL short-circuits the DB slug resolution entirely.
Made-with: Cursor
* Update waitlist, run ralph experiments
* fix PR files pagination: use octokit.paginate() for >100 files
* fix garbled FAQ answer on landing page
* track cache read/write tokens in OpenCode agent usage
* wrap dispatch() calls in try/catch to prevent webhook retries on transient failures
* replace raw error messages with generic responses in API routes
* guard request.json() calls with try-catch returning 400 on malformed bodies
* log warning when GraphQL review thread/comment counts hit pagination limits
* reduce review comment cache TTL from 24 hours to 10 minutes
* use select instead of include for proxyKey in workflow run queries
* align Claude agent activity timeout to 5 minutes to match OpenCode agent
* add in-memory dedup for PR close webhooks to prevent duplicate indexing
* extract isPullfrogLogin() helper for shared Pullfrog detection logic
* check response.ok on log fetch in checkSuite.ts
* add 10s timeouts to checkSuite API calls and log fetch
* parallelize proxy key usage API calls with Promise.allSettled
* fix three typos on landing page: colleage, dectects, reponse
* move MAX_STDERR_LINES constant to shared.ts
* add indexes on Repo.accountId and PFUser.accountId FK columns
* remove unused Permission enum from Prisma schema
* populate author and keywords in action/package.json
* use crypto.timingSafeEqual for all secret comparisons
* add missing env vars to globals.ts: R2, webhook, and API secrets
* remove commented-out UserRepo model from Prisma schema
* replace console.log/error with log utility in production API routes
* replace catch(error: any) with proper type guards in getUserRole
* remove stale TODO comment on console page
* handle repository_transferred webhook to update owner
* show toast.error instead of console.error on mode/workflow mutation failures
* add Space key handler for keyboard navigation on workflow run links
* replace role=link spans with button elements for proper accessibility
* add root 404 page with Pullfrog branding
* update ISSUES.md: mark completed items
* mark remaining low-priority UX items as addressed
* add error logging alongside toasts, add check script, update ralph commands
* address review feedback: squash migrations, fix try/catch scope, wire up globals consumers
- squash drop_permission_enum migration into add_indexes migration (one migration per PR)
- move getPullRequest() outside try/catch in mention handler so errors aren't mislogged as "dispatch failed"
- restore key ID in proxyKeys.ts Promise.allSettled error log
- remove accidental asdf.txt and ralph.md files
- wire up globals.ts exports to consumers (r2-uploads, r2-private, verifyHookdeckSignature, sync-usage, forwardPreviewWebhook, dispatch-workflow)
Made-with: Cursor
* update model snapshot (qwen3.6-plus-preview renamed to qwen3.6-plus)
Made-with: Cursor
* Update waitlist, run ralph experiments
* improve review quality: add --effort flag, subagent guidance, remove dead prompts
- add --effort high/max to Claude Code CLI (max for Opus, high for Sonnet/Haiku).
default was silently dropped from high to medium in March 2026.
- add subagent guidance to Review/IncrementalReview modeGuidance for parallel
investigation of large cross-cutting PRs (read-only, no side effects).
- remove "THINK HARDER" from mode prompts (vestigial, no longer controls thinking).
- remove redundant mode.prompt bodies from modes.ts — the actual guidance lives in
modeGuidance (selectMode.ts) and mode.prompt was dead code for all built-in modes
since the delegation system was removed in March.
Made-with: Cursor
* make Mode.prompt optional, remove ModeSchema dead code
prompt is only needed by custom user-defined modes (validated by Zod
modeSchema in utils/schemas/modes.ts). built-in modes get their guidance
from modeGuidance in selectMode.ts. the arktype ModeSchema was never
imported anywhere.
Made-with: Cursor
* make modes.ts the single source of truth for mode guidance
move all mode guidance from modeGuidance in selectMode.ts into
mode.prompt in modes.ts. selectMode.ts now only contains the runtime
tool logic (resolving modes, merging user instructions, handling
PlanEdit/SummaryUpdate overrides). this eliminates the confusing
fallback chain where someone editing mode instructions had to know
to look in selectMode.ts rather than modes.ts.
Made-with: Cursor
* add self-review subagent step to Build mode, update wiki
Build mode now delegates a read-only subagent to review the diff
before committing, catching bugs/logic errors/edge cases that the
builder might miss. Also updates wiki/modes.md to reflect the
single-source-of-truth architecture (modes.ts owns all guidance,
selectMode.ts is pure runtime logic).
Made-with: Cursor
* update model snapshot (openrouter qwen3.6-plus rename)
Made-with: Cursor
model resolution was duplicated inside each agent (opentoad, claude) and
PULLFROG_MODEL override was not considered when choosing the agent. now
resolveModel() runs first in main.ts, its result feeds into resolveAgent()
for agent selection, and the resolved model is passed to the agent via
ctx.resolvedModel. agents only handle their own fallback (opentoad: auto-select
via opencode models, claude: strip provider prefix).
also removes the hardcoded anthropic/claude-sonnet test runner default since
ANTHROPIC_API_KEY is no longer in CI.
Made-with: Cursor
- stop capturing/displaying total_cost_usd from Claude CLI (theoretical cost is misleading for subscription users)
- remove Cost column from action logs table and GitHub Job Summary
- extract SecretsCard into its own sidebar tab with KeyRound icon
- remove children prop from AgentSettingsSection
Made-with: Cursor
* feat: add encrypted account-level secrets with UI for adding API keys
Adds AccountSecret model with AES-256-GCM encryption, API routes for
CRUD, "Add secret" button in model costs section, and injects decrypted
secrets into action env (YAML secrets take precedence).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: repo secrets, sidebar icons, lazy learnings history
- add repo-level secrets with inheritance from org secrets
- add icons to console sidebar sections
- fix learnings history modal: lazy fetch with hover prefetch,
strip content from list response, load content per-expansion
Made-with: Cursor
* add input validation bounds for secrets and fix client-side name filter
Made-with: Cursor
* refactor: migrate all client-side data fetching to TanStack Query
Replace manual useState/useEffect/fetch patterns and the custom
usePolling hook with useQuery, useInfiniteQuery, and useMutation
across the entire frontend for consistent caching, background
refetching, and reactive invalidation.
- ActiveWorkflowRunsSection: useQuery + refetchInterval
- WorkflowRunHistory: useInfiniteQuery + polling query
- LearningsSection: useQuery per revision (lazy)
- FlagsSettings: self-contained useQuery + useMutation
- SecretsCard: useMutation for delete
- AddWorkflowButton, VerifyWorkflowButton: useMutation
- EmailSignupForm, email-waitlist: useMutation
- providers.tsx: enable refetchOnWindowFocus
- Delete usePolling.ts (no remaining consumers)
Made-with: Cursor
* address PR review: squash migrations, rename accountSecrets → dbSecrets
Squash the two separate secrets migrations into a single migration.
Rename the wire format field from accountSecrets to dbSecrets since
it now carries merged account + repo secrets.
Made-with: Cursor
* fix: update proxyKeys.ts imports after cache.ts -> yes package migration
Made-with: Cursor
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Claude Code agent for Anthropic model users
Re-adds Claude Code support (removed in #478) so users with Anthropic API
keys or Claude Code OAuth tokens can use their Claude subscriptions directly.
When an Anthropic model is selected and Claude Code credentials are available,
the system auto-selects the Claude agent instead of OpenCode. The harness
mirrors opentoad's security model: native Bash blocked via --disallowedTools,
MCP ShellTool for restricted shell, ASKPASS for git auth. Includes NDJSON
streaming, provider error detection, cache/cost tracking, browser skill,
and todo progress tracking.
Key changes:
- action/agents/claude.ts: full Claude Code harness
- action/utils/agent.ts: auto-select Claude for anthropic/* models
- action/utils/providerErrors.ts: extracted shared provider error detection
- action/utils/skills.ts: extracted shared skill installation (agent-aware)
- action/models.ts: add CLAUDE_CODE_OAUTH_TOKEN to anthropic envVars
- action/utils/docker.ts: add CLAUDE_CODE_OAUTH_TOKEN to test env allowlist
- CI: add claude to test matrix, pass CLAUDE_CODE_OAUTH_TOKEN secret
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove unused toolId variable, fix apiKeys test env cleanup
The apiKeys test cleanup stripped *_API_KEY vars but missed
CLAUDE_CODE_OAUTH_TOKEN which doesn't match that pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: strip provider prefix from PULLFROG_MODEL in Claude agent
the env override path was returning the raw value (e.g.
"anthropic/claude-sonnet-4-5") without stripping the provider prefix,
causing the Claude CLI to receive an invalid model ID.
Made-with: Cursor
* fix: remove dead cliPath field, add CLAUDE_CODE_OAUTH_TOKEN to workflows
remove unused cliPath from Claude agent RunParams, and pass
CLAUDE_CODE_OAUTH_TOKEN through all pullfrog.yml workflow templates
so users with Claude Pro/Team subscriptions can use their membership.
Made-with: Cursor
* fix: block Bash subagent in Claude Code disallowedTools
Made-with: Cursor
* chore: update model snapshot (opencode/openrouter latest → qwen3.6-plus-free)
Made-with: Cursor
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix proxy key usage tracking and add OSS spend reporting
replace ProxyKey.disabled with finalizedAt to fix a race where keys
were disabled before their usage was synced, causing all HWM values to
be zero. retireKey now fetches final usage from OpenRouter and records
it atomically with optimistic concurrency. syncAccountUsage skips keys
that fail to fetch rather than recording false zeros.
other fixes:
- wrap OpenRouter API calls in retry logic (exponential backoff)
- reconcileStaleWorkflowRuns now retires proxy keys for completed runs
- subprocess activity timeout only tracks stdout (stderr retry loops
no longer prevent hung agent detection)
- add oss-spend script (single bulk fetch from OpenRouter) and
backfill-proxy-key-usage script
Made-with: Cursor
* address review: move isActiveKey check inside transaction, remove redundant guard
Made-with: Cursor
