* eager signup credit + free-OpenCode fallback when BYOK has no key
addresses the silent-churn pattern that took out 15 first-run-failure
accounts post-launch: GH Actions secret references resolved to empty
strings (because the secrets didn't exist on the repo), the action
launched Claude Code with no key, the LLM provider 401'd, and the run
died in seconds with a synthetic "Invalid API key" message. those
accounts had no Router credits to fall back to because the lazy claim
required a dashboard visit they never made.
three changes, one PR:
1. Eager $10 signup credit at account creation. Both account-creation
sites (`upsertAccountByClerkId` for dashboard signin, `fetchOrCreateRepo`
for CLI / GH-App-only) now insert the `CreditGrant { reason: "signup" }`
in the same transaction as the `accounts` row. CLI installers who
never sign in get the credit. The dashboard `/signup-credit/claim`
POST stays as an idempotent backstop for accounts created before
this shipped.
2. Free-OpenCode fallback in the action. When the configured BYOK slug
needs a provider key the runner doesn't have, swap to
`opencode/minimax-m2.5-free` before agent selection so the run still
succeeds. Surfaced via a `» fell back from <slug> to <free>` warning
in the action log. Skipped on Router runs (Pullfrog mints the key)
and when no model is configured (auto-select-with-throw still fires
for the genuinely-misconfigured case).
3. New action-test fixture `byok-no-keys-fallback` that empty-strings
every known provider key (matching how GH Actions handles missing
secrets) and asserts the run succeeds with the fallback log line
present. plus a unit test for the helper covering each skip case.
skipping the schema flip from `byok` to `router` — that's coming via
the onboarding-stepper PR (#762).
* fallback: skip Bedrock + surface in PR-comment footer
addresses copilot review on #789 (real bug — parseModel throws on
Bedrock raw IDs that have no slash, would crash before
validateBedrockSetup could surface its own error) and the user-side
ask to make the fallback visible in PR comments.
- selectFallbackModelIfNeeded skips when resolvedModel has no '/' so
Bedrock routing IDs (e.g. us.anthropic.claude-opus-4-7) don't crash
inside hasProviderKey -> parseModel. unit test covers it.
- toolState.modelFallback records the configured slug we fell back
from. set in main.ts when fallback engages.
- buildPullfrogFooter accepts fallbackFrom and renders
"Using `MiniMax M2.5` (free) (credentials for Claude Opus not
configured)" so the substitution is visible in PR comments,
reviews, PR bodies, and error reports.
- threaded through all four action-side footer call sites
(mcp/comment, mcp/pr, mcp/review, utils/errorReport). server-side
call sites in triggerWorkflow.ts / handleWorkflowRunWebhook.ts
fire pre-action and don't have toolState — left as-is.
* fallback footer: use provider display name + document email asymmetry
addresses pullfrog reviewer findings on #789:
- footer now shows 'credentials for Anthropic not configured' (provider
display name from `providers.anthropic.displayName`) instead of the
per-model name. credentials are provider-scoped (ANTHROPIC_API_KEY
covers all Anthropic models), so this matches what the user actually
needs to fix.
- document the intentional asymmetry between eager and lazy signup
credit paths: eager skips both the signupCreditClaimedEmail and the
per-grant team@ alert. comment explains why (the 'new account
created' alert already covers it on the eager path; the user-facing
email assumes a user-initiated action that hasn't happened yet for
CLI/GH-App-only signups).
- skipping the backfill for the 15 historical accounts per user's
earlier decision — they all uninstalled, so the cohort self-selected
out of being reachable.
* fallback: gate on resolvedModel + skip resolveModel re-resolve post-swap
local agnostic fixture run surfaced two real bugs the unit tests didn't
catch:
1. fallback gate was on configuredSlug (=payload.model) but the test
uses PULLFROG_MODEL to set the model, which is read by resolveModel
AFTER its slug arg. configuredSlug stayed undefined → fallback never
fired. drop configuredSlug from the helper signature; gate purely on
resolvedModel since that's the same value regardless of how the
model was specified (DB config vs PULLFROG_MODEL env).
2. when fallback engaged, the post-swap resolveModel({slug: fallback.to})
call was ALSO honoring PULLFROG_MODEL, re-overriding the fallback
target back to the unkeyed model. validateAgentApiKey then threw
"no API key found" against the original model. fix: skip the
re-resolve. fallback.to is already a CLI-ready specifier.
unit tests updated for the new helper signature (8 tests, all pass).
fallback log line confirmed emitted in the local run pre-second-fix;
the second fix unblocks the validation that previously threw.
`toolState.model` was set only to `payload.model` (the stored slug, often
undefined for router/oss runs that derive the target from `proxyModel`).
the footer's "Using `…`" segment is gated on a truthy model, so router
runs on repos without an explicit model setting shipped reviews/comments
with no model badge — e.g. PR #614's review showed no model despite
running `openrouter/anthropic/claude-opus-4.7` via proxy.
now mirror the priority used by `resolveModelForLog` and `isGeminiRouted`:
`payload.proxyModel ?? resolvedModel ?? payload.model`. also reverse-look
up by `resolve`/`openRouterResolve` in `formatModelLabel` so a proxy
target like "openrouter/anthropic/claude-opus-4.7" still renders as
"Claude Opus".
* bump model registry for deepseek v4, kimi k2.6, claude opus 4.7
deepseek released v4 (pro/flash) on 2026-04-24 as a generational replacement
for v3-era reasoner/chat. deepseek will fully retire deepseek-chat and
deepseek-reasoner on 2026-07-24 — both already route server-side to v4-flash.
introduce deepseek-pro (preferred) and deepseek-flash slugs and mark the
legacy aliases deprecated via fallback so existing users transparently
upgrade. mirror on the openrouter side.
also bump moonshotai/kimi to k2.6 (from k2.5, 2026-04-21 release) and bump
the anthropic claude-opus openrouter resolves to 4.7 (we'd already moved the
native side to claude-opus-4-7 but openrouter resolves still pointed at 4.6).
update OSS_PROXY_MODEL fallback and stale doc reference accordingly.
snapshot regenerated; all 111 catalog tests + 66 unit tests pass.
* walk fallback chain when resolving the OSS proxy model
the OSS proxy path in run-context/route.ts read alias.openRouterResolve
directly, bypassing the fallback chain. so an OSS repo configured with
deepseek/deepseek-reasoner kept proxying to openrouter/deepseek/deepseek-v3.2
instead of resolving through the new fallback to openrouter/deepseek-v4-pro.
that worked today (v3.2 routes server-side to V4-Flash) but breaks when
deepseek and openrouter retire v3.2 alongside the 2026-07-24 deprecation.
extract the chain walk into a private resolveTerminalAlias helper and add
resolveOpenRouterModel that mirrors resolveCliModel but returns
openRouterResolve. fallback semantics now apply uniformly across both
runtime resolution paths.
* hide deprecated aliases from model selector dropdowns
aliases with a fallback (currently deepseek-reasoner / deepseek-chat /
openrouter/deepseek-chat) should not be selectable from the model dropdown
or the interactive cli model picker — they're a transition path, not a
choice. but if a repo already has a deprecated slug stored in the db, the
selector trigger still resolves it against the full alias registry so the
display name renders correctly until the user opens the menu and picks a
new model.
verified manually: deepseek submenu shows pro+flash only, openrouter submenu
shows pro+flash but no chat, and a deprecated stored value still renders
its full display name in the trigger.
* ci: run models-live on PRs that touch resolution files
Previously the per-alias smoke matrix only fired on push-to-main, so
resolution-affecting PRs (this one included) shipped without ever
exercising the agent harness against the real provider for each alias.
Loosen the gate on the `aliases` step in the `changes` job to fire
whenever the `models` paths-filter matches (action/models.ts,
action/package.json, action/agents/**) — same set that already drives
the comment about "resolution-affecting files". `models-live` itself
is unchanged: it still keys on a non-empty matrix.
`models-catalog` stays gated to main-push intentionally — its existing
comment justifies that (transient upstream catalog drift shouldn't
block PRs).
* relabel codex aliases as GPT, bump to 5.5 family, add gpt-pro
OpenAI retired the "-codex" model suffix on 2026-07-23 (gpt-5.3-codex,
gpt-5.1-codex-mini, gpt-5.2-codex et al all shut down) and unified the
codex+gpt lines into a single family at gpt-5.4. Per OpenAI's own
deprecation table, every "-codex" substitute is plain gpt-5.x — no
future Codex-suffixed frontier models are coming.
Keep the existing slugs for DB stability (no migration needed) but roll
displayName + resolve forward across openai, opencode, and openrouter:
- openai/gpt-codex → "GPT" → openai/gpt-5.5
- openai/gpt-codex-mini → "GPT Mini" → openai/gpt-5.4-mini
- openai/gpt-pro (new) → "GPT Pro" → openai/gpt-5.5-pro
Same relabel + new gpt-pro slug for opencode/* and openrouter/*.
gpt-5.5 (and gpt-5.5-pro) hit the OpenAI public API on 2026-04-24,
day after launch — both are live on OpenRouter as well.
There's no gpt-5.5-mini yet (analysts speculate late June – mid August
based on the gpt-5.4-mini cycle), so "GPT Mini" stays at gpt-5.4-mini
for now; one-line bump when the smaller variant ships.
Also pick up unrelated upstream catalog drift in the snapshot
(xai/grok-4.3 released 2026-05-01, openrouter/poolside laguna).
* deprecate gpt-codex aliases, mint gpt/gpt-pro/gpt-mini, render terminal alias in UI
The previous commit relabeled gpt-codex/gpt-codex-mini in place ("GPT" /
"GPT Mini") so a single slug carried two different identities. That worked
but was self-contradictory: the slug name no longer described the model.
Switch to the same shape we use for the deepseek V3→V4 transition:
- Mint new live slugs: openai/gpt, openai/gpt-pro, openai/gpt-mini
(mirrored on opencode/* and openrouter/*)
- Restore honest deprecated state on gpt-codex/gpt-codex-mini —
displayName "GPT Codex" / "GPT Codex Mini", original 5.3-codex /
5.1-codex-mini resolves, fallback set to the new gpt / gpt-mini slugs
- resolveCliModel + resolveOpenRouterModel walk the chain (existing
machinery), so DB rows holding "openai/gpt-codex" transparently route
to gpt-5.5 with no migration
UI render contract: display sites resolve to the *terminal* alias so a
deprecated stored slug shows the model the user is actually running, not
the historical name. Three call sites updated:
- components/ModelSelector.tsx (dropdown trigger label + provider label)
- action/utils/buildPullfrogFooter.ts (PR-comment "Using `X`" footer)
- action/commands/init.ts ("using model X" startup line)
Promoted internal resolveTerminalAlias → exported resolveDisplayAlias so
all three sites use the same primitive (also re-exported from external.ts
+ internal/index.ts so the Next.js app can import it).
Selectable lists (dropdown options, init picker) still filter on
!a.fallback so deprecated slugs never appear as fresh choices — only
deprecated stored values render.
wiki/model-resolution.md: replaced the muddled "slug names outlive
product names" bullet with a clear decision table for in-place bump
(generational, e.g. Opus 4.6 → 4.7) vs. deprecate+replace (vendor
restructures, e.g. codex → unified GPT, deepseek V3 → V4). Documents
the UI render contract too.
models-live CI matrix will smoke-test all 6 new slugs (gpt, gpt-pro,
gpt-mini × openai/opencode/openrouter) plus the 6 deprecated codex slugs
(which resolve through fallback to the same terminal targets) — 12 jobs
total against real provider APIs.
* wiki: slugs are evergreen, resolves are versioned
Document the slug-naming rule explicitly so future entries don't repeat
the deepseek-chat/deepseek-reasoner mistake (mirroring an upstream's
versioned/product-line-specific ID into the slug). Slugs should track
brand-style tier names that survive major version bumps; embedding
versions is the resolve string's job.
* add wiki/betterstack.md documenting log querying, request-ID grouping, and MCP usage
Made-with: Cursor
* fix webhook race conditions: separate runId assignment from data updates
the workflow_run webhook handler had a race where concurrent handlers assigned
the same runId to different pending records. the loser's P2002 silently dropped
data updates (jobId, status, completedAt). fix by splitting into two steps:
assignRunId() handles the race-safe unique assignment, then data updates always
target where: { runId } so they hit the correct record regardless of who won.
also downgrade R2 ObjectLockedByBucketPolicy errors from error to warn level
since duplicate webhook deliveries writing the same key is expected under load.
Made-with: Cursor
* homepage copy refresh + fix skills CLI installation
- update hero to "Agent x GitHub" with new subtagline
- rewrite intro paragraphs: workflow, harness capabilities, billing
- add feature sections: bash isolation, headless browser, MCP tools
- update FAQ answers, footer attribution, free-for-oss copy
- update APP_DESCRIPTION for SEO
- fix skills install: use npx from tmpdir instead of local binary
(the bundled action has no node_modules; running npx from tmpdir
avoids project .npmrc with pnpm settings breaking binary resolution)
- instruct agents to use markdown image syntax in upload_file tool
- start dependency installation eagerly from main.ts
- include event title in task instructions
Made-with: Cursor
GitHub markdown needs a blank line between </details> and subsequent
HTML elements. Without it, the footer renders inside the collapsed
section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* show model name in footer, drop pullfrog.com link
add model slug to buildPullfrogFooter so every Pullfrog comment
displays the active model (e.g. "Using `Big Pickle` (free)" or
"Using `Claude Opus`"). remove the pullfrog.com link from all footers.
Made-with: Cursor
* reject <br/> tags in comment bodies, add prompt guidance
add runtime validation in addFooter that throws if <br/> is followed
by a non-blank line (breaks GitHub heading rendering). the agent sees
the error and retries with clean markdown. also update Summarize mode
prompt to explicitly forbid <br/> tags.
Made-with: Cursor
* fix <br/> guidance: move to event instructions, clarify blank line rule
the formatting rule belongs in DEFAULT_PR_SUMMARY_INSTRUCTIONS (event
instructions), not the Summarize mode prompt. clarify that <br/> must
always be followed by a blank line before headings.
Made-with: Cursor
* generalize block-level HTML rule in summary instructions
add a prominent top-level rule about requiring blank lines between ALL
block-level HTML elements and markdown syntax, not just <br/>.
Made-with: Cursor
* move model to toolState instead of threading through params
model is set once at startup and read everywhere — it belongs on
toolState, not threaded as a separate param through 8 call sites.
postCleanup runs without toolState so it just omits the model label.
Made-with: Cursor
* update models.dev snapshot (openai latest changed)
Made-with: Cursor
* add comment to models snapshot test explaining its purpose
Made-with: Cursor
* agent & model refactor with ASKPASS git auth, UI restructure, clerk v7
Made-with: Cursor
* fix stale agent/effort refs, add tests for askpass + model resolution
- reviewCleanup.ts: payload.agent -> payload.model, remove effort
- selectMode.ts PlanEdit: remove delegation/subagent/effort references
- pullfrog.yml.ts: update env vars (drop GOOGLE_API_KEY/CURSOR_API_KEY,
add GOOGLE_GENERATIVE_AI_API_KEY/XAI_API_KEY/MOONSHOT_API_KEY/OPENCODE_API_KEY)
- FlagsSettings/RepoInstructionsSection: remove stale effort/timeout copy
- new: gitAuthServer.test.ts (10 tests — lifecycle, token delivery, tamper detection, script gen)
- new: agent.test.ts (4 tests — default opentoad, AGENT_OVERRIDE, invalid override)
- new: models.test.ts (19 tests — parseModel, resolution, registry invariants)
- update models.dev snapshot
Made-with: Cursor
* fix changed-agents.sh to filter legacy agent files from CI matrix
legacy agent files (claude.ts, codex.ts, etc.) are @ts-nocheck and not
exported from index.ts. changed-agents.sh now reads index.ts imports to
build the active agent set and treats changes to inactive files as
non-agent changes (opentoad canary only).
Made-with: Cursor
* remove MCP file tools, old agent harnesses, and obsolete security tests
ASKPASS-based git auth makes the old MCP file tool security layer unnecessary:
- token never in subprocess env, so symlink/gitattributes/hook attacks can't exfiltrate it
- agents now use native file tools (OpenCode builtin read/edit)
deleted:
- action/mcp/file.ts (file_read, file_write, file_edit, file_delete, list_directory)
- action/mcp/index.ts (dead re-export)
- agent harnesses: claude.ts, codex.ts, cursor.ts, gemini.ts, opencode.ts
- opencode-runner.ts (inlined into opentoad.ts)
- security tests that validated MCP file tool restrictions
- commented-out three-step review flow (~300 lines)
- sanitizeSchema/wrapSchema dead code from mcp/shared.ts
- OPENCODE_MODEL_MINI/MAX env vars (effort-level model overrides removed)
updated test prompts to use generic file ops instead of MCP tool names.
restored pkg-json-scripts + requirements-txt-attack (test --ignore-scripts defense).
Made-with: Cursor
* bump actions/checkout v4 → v6 (node 24)
node 20 actions deprecated june 2, 2026.
Made-with: Cursor
* temporarily disable fail-fast on agnostic tests to debug checkout@v6
Made-with: Cursor
* re-enable fail-fast on agnostic tests
Made-with: Cursor
* fix test token mismatch: mint OIDC tokens scoped to target repo
CI tests override GITHUB_REPOSITORY to pullfrog/test-repo but inherit
the runner's GITHUB_TOKEN (scoped to pullfrog/app), causing 401s on
every run-context fetch. Clear GITHUB_TOKEN in the test subprocess so
ensureGitHubToken() mints a properly scoped token via OIDC.
Also centralizes the default GITHUB_REPOSITORY in runAgentStreaming
instead of repeating it in every test file, and fixes preview-cleanup
to remove workers from all queues (not just name-matching ones).
Made-with: Cursor
* fix ensureGitHubToken to try OIDC when app credentials are absent
ensureGitHubToken only attempted token minting when GITHUB_APP_ID and
GITHUB_PRIVATE_KEY were set. In CI, OIDC is available but app creds
aren't exposed — so the guard prevented minting entirely.
Made-with: Cursor
* dead code cleanup: remove remnants of deleted agents, file tools, effort system
remove unused @anthropic-ai/claude-agent-sdk and @openai/codex-sdk deps,
orphaned file-tool security tests, dead GEMINI_MODEL passthrough, stale
opencode-runner wiki refs, deleted test file references, and MCP file tool
docs. rename docs/effort → docs/models. fix vitest setup: move dotenv to
globalSetup (runs once before forks instead of per-file, 19s → 200ms).
Made-with: Cursor
* address review feedback: remove dead code, update stale references
- remove AGENT_OVERRIDE (only opentoad exists)
- remove shellToolName plumbing (always restricted shell)
- bump action version to 0.0.179
- remove CURSOR_API_KEY from all workflows/configs
- remove OPENCODE_MODEL_MINI/MAX from workflows/docs
- delete wiki/effort.md, rewrite docs/effort.mdx as "Models"
- rewrite wiki/modes.md: orchestrator/subagent → single agent
- simplify flag system: drop builtin flag extraction (debug, effort,
timeout, agent), keep custom flag replacement only
- reserve all legacy flag names to prevent custom flag conflicts
Made-with: Cursor
* regenerate lockfile after removing claude-agent-sdk and codex-sdk
Made-with: Cursor
* fix import ordering, add lockfile check to pre-push hook
Made-with: Cursor
* remove dead debug payload field, stale packageExtensions
Made-with: Cursor
* merge proc-sandbox and token-exfil into a single test
proc-sandbox and token-exfil were duplicative — both tested that
SANDBOX_TEST_TOKEN couldn't be exfiltrated. consolidated into
token-exfil with shell:restricted (which actually exercises filterEnv)
and the /proc attack vector hints from proc-sandbox.
Made-with: Cursor
* fix wiki adversarial.md to match actual tokenExfil validator
Made-with: Cursor
* add "Rerun failed job" link to error comment footer
* Remove issueNumber guard from rerun link
The rerun action only needs run_id — the issue number in the trigger
URL path is just a route segment requirement. Use 0 as a fallback
so the link is always shown when a run ID is available.
* Overload `[number]` path segment as `runId` for the rerun action
For the rerun trigger, the `[number]` path param now carries the
workflow run ID instead of an issue number. The rerun link changes
from `/trigger/o/r/ISSUE?action=rerun&run_id=RID` to
`/trigger/o/r/RID?action=rerun`.
- Remove `run_id` query param from page.tsx and searchParams type
- Add error handling around `reRunWorkflow` for invalid run IDs
- Drop `issueNumber` from `BuildErrorCommentBodyParams` and all
rerun link builders (errorReport, exitHandler, postCleanup)
* Reorder validation: action first, then rerun, then issueNumber
* address review: remove early toolState assignment, restructure rerun into dedicated block
* revert self-contained rerun block, share auth logic via issueOrRunId
* improve error handling
* normalize runid early
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com>
Colin McDonnell