Files
shockbot/utils/agentHangReport.ts
T
David Blass 4d1fd5ea1a fix: 4 unaddressed log-audit / run-audit findings + close 10 already-resolved issues (#785)
* fix: 4 unaddressed log-audit / run-audit findings

closes 4 issues with code changes; 7 issues are already addressed by #769
and 3 are deferred — see PR description.

#782 Anthropic 401 → `isApiKeyAuthError` now matches the direct-Anthropic
401 shape (`Failed to authenticate. API Error: 401 ...`,
`authentication_error`, `Invalid bearer token`, `api_error_status=401`)
so revoked / mistyped / rotated `ANTHROPIC_API_KEY` users see the
formatted rotate-key CTA instead of a raw 401 JSON dump.

#778 billing-class provider errors → `providerErrors.ts` now classifies
`CreditsError` / `FreeUsageLimitError` / `Insufficient balance` /
`spending cap` as `provider billing exhausted` *before* status-code
patterns can win and tag them as transient `auth error (401)` /
`rate limited (429)`. `agentHangReport.ts` swaps the bare
"Pullfrog stalled — auth error" headline for a billing-specific CTA
(extracts the provider's billing URL when present).

#775 silent IncrementalReview swallows `BillingError` →
`reportErrorToComment` now optionally falls through to creating a fresh
issue comment on `toolState.issueNumber` when no progress comment
exists. Wired with `createIfMissing: true` from the `BillingError` /
`TransientError` paths in `proxy.ts` so silent triggers
(`pull_request_synchronize`) finally surface the router-balance signal
on the PR instead of only in the GH job summary.

#773 `currentUser()` inside `after()` →
`fillInstallerIdentityIfMissing` is split into
`resolveInstallerIdentity` (must run inside the request body) and
`fillInstallerIdentity` (DB-only, safe in `after()`). The
`/console/[owner]` caller now resolves Clerk identity up-front and
defers only the prisma write, fixing the broken installer-identity
backfill on org-console first-admin visits.

Co-authored-by: Cursor <cursoragent@cursor.com>

* add /audits cursor command for triaging run-audit + log-audit issues

Co-authored-by: Cursor <cursoragent@cursor.com>

* review prompt: tighten body-section bar + inline technical-details (#770)

* review prompt: tighten body-section bar + add inline technical-details

Two layers of tightening to the Review/IncrementalReview prompts in
PR_SUMMARY_FORMAT (and the per-mode aggregate-&-draft step):

1. Reframe inline-vs-body split. Body `### ` sections are now reserved
   for concerns that genuinely have no line to anchor to — absence,
   sequencing, design decisions, scope questions, architectural risk.
   Drop the "cross-cutting concerns" framing (misled the agent into
   either filing nothing in the body or filing multi-file anchored
   findings there).

2. Add a "Hunt for non-anchored concerns" sub-step to both Review (step
   6) and IncrementalReview (step 8) aggregate phases. Diagnosis from
   PR #767's auto-review: on substantial PRs the agent surfaced
   findings but routed all of them inline, producing reviews with zero
   `### ` body sections even on diffs where non-anchored concerns
   clearly existed.

3. Replace the abstract `### ` example with a concrete non-anchored
   one ("Legacy `opencode.ts` has no documented deletion plan") so the
   agent pattern-matches the absence-shaped finding, not a line-bug.

4. Add an "Inline technical details" subsection to PR_SUMMARY_FORMAT
   so inline comments can carry a `<details>Technical details</details>`
   block when the fix has cross-file implications. Rename the existing
   "Agent details" inline collapsible to "Technical details" for
   consistency with body sections.

5. (Carried over from prior uncommitted work) Restructure the review
   metadata block from `<details>Review metadata</details>` into an
   HTML comment + an italic TL;DR commit-range line. The HTML comment
   keeps the metadata addressable for downstream agents without
   eating user-visible review real estate.

No tests touched.

* wiki: document multi-model end-to-end eval pattern

* feat(promo): cookie-stashed promo codes for onboarding rewards (#771)

* feat(promo): cookie-stashed promo codes for onboarding rewards

Operator hands out a link like https://pullfrog.com/start?promo=FROGGY;
middleware validates the code against an in-code registry, stashes it in
an HttpOnly cookie, and the install callback applies the reward once
the GH-side account exists. v1 reward: unlimited_runs (lifts the monthly
free-runs cap to 1M, same convention prod-grandfathered accounts use).

No schema changes. Idempotent across reinstalls via the lte: 100 gate.

* fix(promo): integrate handler into existing proxy.ts (Next 16 rename)

* docs(promo): clarify sentinel + sync plan doc with renamed paths

* feat(promo): add FOUNDATIONS code

* feat(promo): show applied promo code in console

* refactor(promo): move cookie set to client-side

* docs(promo): point JSDocs at PromoCookieSetter, not proxy.ts

* billing: cap counts only successful runs (#787)

* billing: cap counts only successful runs

`reserveRun` was counting `WorkflowRun` rows regardless of status against
`Account.includedMonthlyRuns`. Failed / cancelled / skipped / timed-out
runs consumed cap slots even though their `billableCents` got zeroed on
the completion webhook — pushing paying users into billable territory
earlier than the contract implies. `inthhq` paid for 2 extra runs this
month because 2 failed runs ate 2 of their 100 free slots.

Cap query now filters on `CAP_CONSUMING_STATUS = "success"`. Only
runs that actually deliver value consume slots; in-flight (`running`)
runs hold no slot until they terminate as success (burst-bypass risk
is theoretical given GH Actions concurrency limits).

Shared constant lives in `utils/billing.ts` and is used in lockstep by
three call sites: `reserveRun` (live cap gate), the billing API's
`runsThisMonth` (dashboard progress bar), and the billing-report
script's `cap` column. Script's `cap` cell was also broken
independently — it compared `monthBillableRuns` (overage count) against
`includedMonthlyRuns` (free cap), so `inthhq` rendered as `125/100
(over)` when the meaningful ratio is `223/100 (over)`. Fixed to use
`mRuns/cap`, which is the same predicate the live billing path uses.

* move CAP_CONSUMING_STATUS to workflowRunStatus.ts + wire script through it

Per copilot review: the JSDoc claimed the billing-report script used the
constant in lockstep, but the script kept `status: "success"` inline. The
script imports from raw-node ESM and can't pull in `next/server`, so it
couldn't import from `utils/billing.ts`. Moved the constant to
`utils/workflowRunStatus.ts` (already Next-free, already the home of
`CONCLUSION_VALUES`) and updated all three call sites to import from
there. Script's `mRuns` query now uses `CAP_CONSUMING_STATUS` directly,
making drift impossible.

* learnings: audit fixes — preamble in TOC, server-side line-boundary truncation, empty-repo intro (#743)

* learnings: surface preamble in TOC, mirror line-boundary truncation server-side, fix empty-repo intro copy

three audit fixes on top of the recent learnings overhaul (#717):

- `parseLearningsHeadings` now prepends a synthetic `(preamble)` entry
  when a body has non-whitespace content before the first heading. the
  prompt instructs the agent NOT to slurp the whole file when a TOC is
  present, so without this any preamble lines were silently invisible
  (realistic transitional case: an agent partially restructures a
  legacy free-text body and leaves bullets above the first `## `).

- server-side PATCH route now applies the same line-boundary-aware
  truncation as the action (defense in depth via a shared
  `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` exported from
  `action/internal`). the raw `.slice` it used before could leave a
  mid-heading tail on any caller that bypassed the client-side
  truncate, breaking the next-seed TOC parse. removes the duplicated
  cap constant.

- `buildLearningsSection` intro no longer asserts "accumulated by
  previous agent runs" — false for fresh repos with zero history. new
  copy is tense-neutral and works for empty + populated bodies. also
  nudges the agent to re-read after mid-run edits (the inlined TOC
  ranges are a run-start snapshot).

Co-authored-by: Cursor <cursoragent@cursor.com>

* learnings prompt: tighten to single evergreen test, allow tool-quirk bullets when they prevent repeat waste

The blanket "no pullfrog tool quirks" ban was wrong — if the agent burned
calls discovering a quirk this run, recording the workaround prevents the
next run from repeating the waste. Reframe around one litmus ("would a
future run do its work better because this bullet exists?") and trust it
to subsume the scattered don'ts. Drop the 3+ months timeframe (arbitrary)
and the four-example pullfrog/PR/date/play-by-play list (the rule
underneath is "don't anchor facts to repo state that will move"). Cuts
~10 lines from a prompt the model was already mostly ignoring; the
remaining anchor list is narrower and more enforceable.

* audit-learnings-r2: align wiki + tighten re-read nudge

- wiki/prompt.md described the post-run reflection prompt as "bans pullfrog-tool quirks (those belong in tool descriptions, not per-repo learnings), bans PR/review/commit/date references" — that's stale after the prompt rewrite. update to: single-litmus framing, expanded anchor list (now includes version pins + line numbers), and explicit allowance for tool-quirk workarounds when discovery burned calls.
- buildLearningsSection re-read nudge said "re-read after editing" which can be read as "re-read the section you edited". in fact any edit shifts the line numbers of every later section in the TOC, not just the edited one. tighten to make that explicit. mirror the new wording in the wiki example block. update the test substring assertion accordingly.

* postRun: refresh JSDoc to match the reflection prompt rewrite

`buildLearningsReflectionPrompt`'s JSDoc still listed "PR-/review-/commit-/date-anchored facts" and "rediscovery of pullfrog-tool quirks" as failure modes the prompt pushes back on. after b586b4f8 the prompt no longer bans tool-quirk bullets (it explicitly allows them when the agent burned calls discovering the quirk), and the anchor list expanded to cover branch refs, version pins, and line numbers too. update the JSDoc so it describes the prompt that actually exists, and call out the cross-repo drift tradeoff that comes with allowing tool-quirk bullets.

* fix(mcp/issueEvents): narrow event.event before Set.has lookup

octokit's listEventsForTimeline union includes timeline-event members where `event` is `event?: string`. `("event" in event)` does not narrow that property to non-undefined, so `relevantEventTypes.has(event.event)` was passing `string | undefined` to a `Set<string>.has`. typescript only flagged this once `cf-worker-indexing` started seeing the file via the type graph that now reaches mcp through the new `truncateAtLineBoundary` re-export in `action/internal/index.ts`. fix the latent bug at the source: require `typeof event.event === "string"` before the Set lookup.

* learnings: split truncation helpers into MCP-free module

re-exporting `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` from `action/utils/learnings.ts` through `action/internal/index.ts` accidentally pulled the entire MCP type graph into the SDK barrel: `learnings.ts` imports `ToolContext` from `mcp/server.ts`, which transitively wires every tool module under `action/mcp/` into anything that imports from `pullfrog/internal`. for `cf-worker-indexing/tsconfig.json` (`customConditions: ["@pullfrog/source"]`) and the root `tsc` (which compiles the proprietary app routes that import from `pullfrog/internal`), this expanded the type-checked surface and surfaced two latent issues in unrelated files (`mcp/issueEvents.ts`, `utils/subprocess.ts`). a 6-line pure string helper has no business dragging mcp/server.ts into anyone else's type graph.

move both symbols to `action/utils/learningsTruncate.ts`. `learnings.ts` re-exports them so existing callers keep working; `internal/index.ts` re-exports from the truncate-only module so the SDK barrel stays MCP-free.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>

* trim first-run celebration email to short personal note

drops the feature-dump bullet list (custom review instructions, github
iteration walkthrough, security model) — wrong moment to teach. keeps
the congrats, the reply CTA, adds discord/x links, keeps the router
credit P.S. handler no longer needs the workflowRun→repo lookup.

* signup-report: per-bucket histogram

Adds a UTC-aligned signups-per-bucket histogram between the overview
block and the company-email list. Empty buckets are pre-filled with 0
so dry spells render as gaps. New `BUCKET=hour|day` env flag with a
smart default (hour if window ≤ 48h, else day). Histogram is also
included in the JSON payload under `histogram: [{key, count}, ...]`.

* signup-report: drop hourly bucket, day-only histogram

* feat(billing): monthly Router spend limits (hard-cap + alert-only) (#748)

* feat(billing): monthly router spend limits (hard-cap + alert-only modes) (#660)

Per-account ceiling on the sum of `router_topup` invoices (pending +
succeeded) for the current UTC calendar month. Closes a gap where a
runaway agent loop, leaked PR trigger, or stuck workflow could
auto-reload indefinitely with no aggregate per-month ceiling.

Two enforcement modes via `RouterLimitMode` enum:
  - `hard_cap`: refuse new auto-reloads; PR comment via reserveRun;
    402 `router_monthly_limit` from /api/proxy-token; email + banner
  - `alert_only`: auto-reload keeps flowing; email + banner only,
    first breach per UTC month

Enforcement is split across reserveRun (pre-dispatch paywall comment)
and /api/proxy-token phase-1 (mid-run 402). Both surfaces read through
the same `getRouterSpentThisMonthCents` helper so the dashboard, the
dispatch gate, and the auto-reload gate can't disagree.

Email dedup uses `Account.routerLimitNotifiedMonth` (YYYY-MM string),
claimed atomically inside the phase-1 SERIALIZABLE txn so concurrent
reloads breaching together send exactly one email. Read-time
comparison with the current month re-arms on rollover — no cron.

Admin surface: `RouterLimitBanner` (reuses `DelinquencyBanner` shell)
above the Router/BYOK tabs in `ModelAccessCard`, with a popover
"Adjust limit" form that PATCHes the existing
/api/account/[owner]/billing/settings route. Same `assertBillingAdmin`
gate that owns the other billing settings — no new auth surface.

See wiki/billing.md § Router monthly spend limit for the full
contract + edge cases.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): anneal pass on monthly Router spend limit (#660)

Round-1 review across 5 lenses (billing-subsystem, correctness,
security, operational-readiness, research-validated-assumptions)
surfaced one critical + three actionable major findings on top of
[#748](https://github.com/pullfrog/app/pull/748).

**Critical — CAS never matched NULL.** `claimRouterLimitNotificationSlot`
used Prisma `NOT { routerLimitNotifiedMonth: monthKey }`, which compiles
to `field != value` — UNKNOWN (not TRUE) against the post-migration
`NULL` default. First breach for any account would never claim the
slot, never stamp the row, and never fire the email (hard_cap or
alert_only). Replaced with `OR: [{ field: null }, { field: { not:
monthKey } }]`, mirroring the `maybeNotifyLowBalance` pattern.

**Major — email gap on manual-top-up over cap.** Breach email was only
wired through `/api/proxy-token`. A manual `/billing-top-up/<owner>`
that crosses the cap blocks dispatch via `reserveRun` but never hits
proxy-token, so the user got the PR comment but no email. Wired the
CAS + `after(maybeNotifyRouterLimit)` into `reserveRun`'s
PaywallError catch (the SERIALIZABLE txn rolled back when we threw,
so we re-claim with the global client; single-statement CAS is its
own race boundary against concurrent proxy-token claims).

**Major — PR paywall comment leaked $ figures.** `router_limit` body
embedded `($X of $Y)` in a comment visible to anyone with PR read
access (public repos, forks, outside collaborators). Other paywall
types deliberately avoid amounts. Removed; deep link still points to
the authenticated console for the figures.

**Medium — observability.** Added `[router-limit]` structured logs at
the three enforcement sites (proxy-token hard_cap 402, proxy-token
alert_only breach, reserveRun paywall) so on-call can grep "did the
cap fire for customer X this month."

**Medium — customer docs.** Added a `### Monthly spend limit` section
to `docs/billing.mdx` (Mintlify) describing the two modes and the
manual-top-up caveat.

**Doc — refund/dispute interaction.** Documented in `wiki/billing.md`
that the cap inherits the existing webhook semantics: disputed
`router_topup` drops from the sum (cap briefly un-trips); refunds
don't flip status today so refunded top-ups keep counting. Matches
wallet behavior — not redefined here.

Accepted as-is (documented or pre-existing): `after()` reliability vs
stamp-before-send tradeoff, alert_only email fires before Stripe
phase-2, proxy-token reads limit fields outside SERIALIZABLE scope
(brief TOCTOU on admin lowering cap), stale paywall comment on cap
clear, no global kill switch (per-account `alert_only` flip is the
practical kill switch), no audit log on cap changes (no existing
audit infra), action version not bumped (separate release commit).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): anneal round 2 on monthly Router spend limit

Round-2 anneal (billing-subsystem, correctness, research-validated,
user-journey, operational-readiness) surfaced a critical merge conflict
and a handful of major correctness + UX gaps on top of [#748](https://github.com/pullfrog/app/pull/748).

**Critical — merge conflict.** While #748 was open, [#755](https://github.com/pullfrog/app/pull/755) extracted
`formatBillingErrorSummary` from `action/main.ts` to
`action/utils/billingErrors.ts`. The PR's new `router_monthly_limit`
arm still lived in `action/main.ts`. Took main's slim orchestrator
wholesale; moved the arm into the extracted file.

**Major — cap = payments only, not dispatch.** `reserveRun` was
pre-empting all PR-comment / `/trigger` dispatch on `spent >= limit`
regardless of wallet balance, contradicting the cap's positioning as
"ceiling on what you pay." An account with $500 of paid-up wallet and
a breached $100 cap couldn't trigger any new run via the comment
path, while GitHub UI re-runs (which bypass `reserveRun`) succeeded —
surface inconsistency. Deleted the pre-dispatch gate; `/api/proxy-token`
is now the sole enforcement point, refusing only the next auto-reload
that would push past. Wallet credit always drains. Dropped the now-dead
`router_limit` arm in `buildPaywallCommentBody`, the dead
`routerSpentCents`/`routerLimitCents` fields on `PaywallError.detail`,
and the post-paywall email-fire David added — all unreachable.

**Major — split `manual_topup` from `router_topup`.** Manual on-session
top-ups at `/billing-top-up/<owner>` were landing as
`Invoice.kind = "router_topup"` and counting toward the cap. The cap
exists to brake *passive* runaway (auto-reload loops); a manual top-up
is a deliberate click-through that the user owns. Added
`InvoiceKind.manual_topup`, flipped the manual write site +
`createTopUpCheckoutSession` metadata, broadened wallet /
reconcile / billing-report reads to `kind IN (router_topup,
manual_topup)`, and scoped `getRouterSpentThisMonthCents` (the cap
aggregate) to `router_topup` only. Worked example: cap=$300,
reload=$100 → exactly three reloads succeed; a fourth is blocked.
Historical rows stay labelled `router_topup` (no backfill); the
asymmetry is small and accepted since the manual flow only existed
alongside auto-reload for a brief window. Extended the
`invoices_kind_matches_stripe_columns` CHECK so `manual_topup` follows
the same shape as `router_topup` (PaymentIntent-backed, no
stripeInvoiceId); split into a second migration because PG forbids
using a freshly-added enum value in the same transaction.

**Major — email reframed around the triggering reload event.** The
`alert_only` body was reporting a pre-eager-write `spentCents` while
the dashboard reads the post-commit value, so email and dashboard
disagreed by exactly one reload. Both flavors now say "Your most
recent $50 auto-reload brought you over your $300 monthly limit"
instead of a running spent-of-cap total — no reconciliation needed,
no more "you've hit your monthly cap" copy firing for partial breaches
(spent=$80 of $100, reload=$30 would have triggered that wording).

**Major — `/trigger/<owner>/<repo>/<n>` paywall copy.** Hardcoded
"You've used your 30 free runs this month. Add a card to continue at
7¢/run." regardless of `detail.reason`. Branched on `cap` vs
`delinquent` so each paywall surfaces actionable copy with the right
CTA. `router_limit` no longer flows through here (per F4 above).

**Major — RouterLimitBanner.** Added an `isAlertBreached` visual
state (amber palette) so an `alert_only` account at $240 of $200 no
longer renders in the same neutral zinc chrome as a healthy under-cap
account. Updated popover copy to reflect the auto-reload-only scope.

**Medium — paywall log line.** Added `detail.reason` to the
`[Installation X] paywall:` log so on-call grepping for "why was this
paused" can distinguish `cap` from `delinquent`.

**Cleanup.** Dropped dead `utcMonthKey` import + re-export in
`maybeNotifyRouterLimit.ts`. Renamed file-internal `reconcileRouterTopup*`
fns + their reconcile-kind labels to `reconcileTopup*` / `topup_*`
since they now handle both kinds. Updated wiki/billing.md +
docs/billing.mdx + schema doc comments throughout.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(billing): drop routerLimitNotifiedMonth sentinel; rely on Resend idempotency-key

The sentinel was the same anti-pattern as `routerLowBalanceEmailedAt`
sitting next to it — a single-purpose state column on `Account` that
encoded a date as a string and required a custom CAS predicate to
read/write race-safely. Plus it had real holes: Resend send failure
left the sentinel stamped and the account silently un-emailed for the
month (F11), mode flips mid-month didn't re-arm (F9), and cap-lowered
edge cases never fired at all.

Replace it with: fire `maybeNotifyRouterLimit` on every breaching
reload, let the Resend `Idempotency-Key`
`router_limit:<accountId>:<monthKey>:<mode>` collapse repeats inside
Resend's 24h dedup window. Continuously-breaching accounts get ~1
reminder per day; brief Resend outages self-heal because the next
breaching reload re-attempts the send. Mode is in the dedup key so
`alert_only → hard_cap` mid-month re-arms a fresh email with the
appropriate copy.

Drops `Account.routerLimitNotifiedMonth` and
`claimRouterLimitNotificationSlot`; simplifies the proxy-token
phase-1 branch significantly. Net diff is negative LOC and the data
model loses a single-purpose sentinel.

Migration was branch-local — never deployed — so I edited the original
add-cap migration in place to drop the column from the ALTER TABLE
rather than chain a drop-column migration on top. Preview Neon
branches reset automatically on history rewrite per wiki/migrations.md.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): hide RouterLimitBanner when no cap is configured

The banner was unconditionally rendered for every billing-enabled
account, including pure-BYOK admins who never touch Router. They got
"No monthly spend limit / Router has spent $0.00" + a divider as
visual noise on the model access page — basically nagging them to set
a feature they may not want. Running without a cap is valid; we don't
nag.

ModelAccessCard now gates the banner block (banner + dividers) on
`routerMonthlyLimitCents !== null`. RouterLimitBanner drops the
no-limit visual state, the "Set monthly limit" CTA text, and the dead
`hasLimit` branching. Cleaner three-state shape (under cap / amber
breached / brick breached).

Discoverability: no-cap users no longer see a UI affordance to set
one. That's deliberate — the cap is a power-user feature documented
in docs/billing.mdx. If discoverability becomes an ask, we can add a
small inline link inside RouterWalletSection without bringing back
the always-visible banner.

Resolves the only outstanding finding from cursor bugbot's review of
ff5328c (banner-visible-for-byok thread).

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(billing): docs/wiki match new "no banner without a cap" reality

Pullfrog bot review of f7672ca pointed out the customer docs still
told users to "Set the cap from the **Monthly spend limit** banner in
the **Model costs** card" — but after hiding the banner for no-cap
accounts there is no such banner to use until you already have a cap.
Catch-22 for first-time setup.

Rewrote docs/billing.mdx to be self-contained: explain what the cap
is, what the two modes do, what the banner shows *once configured*,
and direct admins to PATCH the billing settings endpoint (or reach
out to support) for first-time setup. Cap is positioned as optional;
running without one is the documented default.

Wiki paragraph in wiki/billing.md updated to match — banner is only
rendered when a cap exists, three visual states (under / amber / red),
no first-time-setup UI nag by design.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): move monthly cap into RouterWalletSection as a normal settings row; drop the banner entirely

The standalone `RouterLimitBanner` was the wrong shape. It only
rendered when a cap was already configured (so there was no UI to
discover the feature in the first place — first-time setup required
hitting the API directly), and it occupied prominent real estate above
the tabs to surface state that already lives in the row's own input
when the form moves down where it belongs.

New shape: monthly cap is just a third row inside `RouterWalletSection`
sibling to **Auto-reload amount** and **Auto-reload threshold**. Gated
the same way (card on file + auto-reload enabled — the only state
where the cap actually means anything). Empty input → no cap, with
placeholder "No limit". Setting a number reveals a **Behavior at
limit** toggle built on the same `Tabs` slider component used for the
Router/BYOK tab switch, so the look matches the rest of the card.

Deletes:
- `RouterLimitBanner` component (212 lines)
- banner mount + conditional + spacers in `ModelAccessCard`
- `AlertTriangle` is still imported (used by `DelinquencyBanner`)

Adds:
- one settings row in `RouterWalletSection` with the cap input + mode tabs
- `routerMonthlyLimitUsd` / `routerLimitMode` plumbed through the
  existing `saveSettings` helper (widened to accept `string | null`)
- `Tabs` / `TabsList` / `TabsTrigger` import

Docs + wiki updated to match the new shape; the customer doc no
longer points at a banner that won't appear.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): split monthly cap input and Behavior-at-limit toggle into separate rows with hr between

Previously bundled both into one row block. Restructure: cap input is
its own row; Behavior-at-limit Tabs gets a sibling row with the
standard `h-5 + hr + h-5` separator between (matching the rhythm of
auto-reload amount → threshold → monthly cap). Mode-toggle row is
gated on `routerMonthlyLimitCents !== null` so the hr + tabs only
appear once a number is in the cap input.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(billing): right-justify Behavior-at-limit Tabs to mirror Auto-reload toggle row

Same `flex items-center justify-between gap-3` layout as the
Auto-reload row: label group on the left, control on the right.
Drops the vertical stack in favour of the horizontal one — looks
identical to the toggle row directly above.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>

* drop italic TL;DR commit-range line from review body

the metadata (sha range, commit list, timestamps) is already in the html
comment for downstream agents. the visible italic line was clutter and
the ellipsis form broke the second sha's auto-link on github anyway.

* add agent-browser fallback rule for unreachable chrome devtools mcp

* onboarding: gated org-console wizard (#762)

* onboarding: gated org-console wizard

Replaces the org console's `/console/[owner]` page with a single-card,
"growing" stepper when the account has zero `Repo` rows. Walks first-time
users through billing mode, BYOK provider+key (if applicable), repo pick,
workflow file creation, and a celebratory redeem-credit moment before
landing them back on the now-populated org console.

## What's new

- New: `components/OnboardingStepper.tsx` — the wizard. Six steps, each
  derived from real persisted state (Account.modelAccessMode,
  AccountSecret, Repo). Step state ladder with progressive disclosure and
  click-to-edit collapsed summaries.
- New: `app/console/[owner]/OnboardingView.tsx` — page-chrome wrapper
  that hosts the stepper inside the same header/sidebar shell as the
  member view.
- Modified: `app/console/[owner]/page.tsx` — adds a `prisma.repo.count`
  gate alongside existing parallel queries; renders OnboardingView when
  count === 0, else falls through to the existing repo grid.

## Schema

- Flipped `Account.modelAccessMode` default from `byok` to `router`.
  Router is the lower-friction default (signup credit funds first ~150
  runs without a card; users can flip to BYOK explicitly via the wizard
  or the existing `<ModelAccessCard>` switch). Existing rows keep their
  current explicit value — Postgres column-default change doesn't
  backfill, by design.
- Migration: `20260516014601_modelaccessmode_default_router`.

## Credit-claim semantics

Killed the historical mount-time auto-claim on `<SignupCreditModal>`.
All claims are now explicit clicks, fired from one of two surfaces:

1. Wizard step 6 "Redeem $10 credit" CTA (Router branch, eligible).
2. New explicit "Redeem $10 credit" button on `<BillingCard>`'s Router
   wallet section, visible only when the new server-derived
   `signupCreditEligible` flag is true (promo active + no prior signup
   or welcome grant). Covers existing users who'd otherwise lose the
   auto-claim entry point.

`<SignupCreditModal>` is now a controlled component (`open` /
`onOpenChange` / `amountCents` props) with a sibling
`useClaimSignupCredit(owner)` hook for explicit invocation. The Sparkles
celebration dialog rendering is unchanged.

## Other touched surfaces

- `app/api/create-workflow/route.ts`: optional `model` body field. When
  present, the route updates `Repo.model` on the row that
  `createWorkflowForRepo` just created/surfaced — wizard threads the
  picked provider's `preferred` model alias through here so a fresh repo
  doesn't sit on null/auto.
- `app/api/account/[owner]/billing/route.ts`: surfaces
  `signupCreditEligible: boolean` (derived from
  `SIGNUP_CREDIT_PROMO_ACTIVE` + grant scan). Drives the new explicit
  redeem button.
- `components/AgentSettings.tsx`: fixes the Router-no-billing copy lie
  ("Runs will draw from your signup credit until exhausted" was false —
  `isInfraCovered` gates Router minting on `hasCardOnFile`, not balance,
  so credit-only-no-card users can't actually spend the grant on Router
  runs). New copy: "Add a card to use Pullfrog Router. Your $10 signup
  credit (if claimed) applies on top."

## Resume-tomorrow detection

Every step's expansion is derived from persisted state (no new column,
no localStorage). With the Router default flip, `modelAccessMode ===
"byok"` is now a reliable signal of explicit user pick, eliminating the
heuristic that the byok-default schema would have required. The only
ambiguous case is "Router-bailed-before-redeem" (looks identical to a
default-Router fresh visit since neither card nor grant exists yet) —
acceptable 1-click cost on revisit.

## Testing

- `pnpm lint`: clean
- `pnpm format`: clean
- `pnpm typecheck`: clean
- `pnpm -C action test`: 596/596 passing
- Visual verification: blocked — Chrome DevTools MCP returned "Not
  connected" across both available servers. Manual walkthrough needed
  before merge to confirm step transitions, going-back UX, and the
  celebration modal redirect destinations match the plan in
  `.cursor/plans/org_onboarding_stepper_4fdfebbb.plan.md`.

* onboarding: drop accordion, multi-repo bulk-onboard, full-width radio rows

Three rounds of UX feedback rolled in:

1. **Drop the accordion.** Steps no longer collapse to a one-line summary
   when "done" — the wizard literally grows by appending steps below as
   the user progresses, and earlier steps stay fully interactive
   (re-flip Router→BYOK, re-pick provider, toggle a repo) without any
   "edit" affordance. `StepShell` now always renders its body for any
   step the user has reached; the only state distinction is the number
   circle (filled = active, check = done).

2. **Step 1 is full-width radio rows, not narrow tabs with side-by-side
   info tiles.** Two rows, each with the option title, an inline
   "Recommended" badge on Router, and a description sentence inside the
   row. The persisted `Account.modelAccessMode` (default `router`)
   drives the initial selection, so step 1 always has one row picked
   on first paint — no "neither selected" empty state.

3. **Multi-repo bulk-onboard.** Step 4 now uses checkboxes; copy reads
   "Select the repos you'd like to install Pullfrog into. We'll create
   a pullfrog.yml GitHub Actions workflow file in each." Step 5 fans
   out N parallel `POST /api/create-workflow` calls (concurrency
   capped at 4) and renders per-repo status inline (running →
   committed / PR #N / already configured / error). Step 6 celebrates
   with a multi-result headline ("Pullfrog is set up across N repos")
   and a sub-line breaking down `committed · PRs awaiting merge ·
   failed` plus a per-repo PR list when any PRs were opened. Single-
   repo path renders the same control surface but with singular copy.

Other bits:
- Per-step description sentences below every title.
- Repo picker shows totalCount inline with the pagination controls and
  "N repos selected" summary below the table.
- Dropped the `userPickedBillingMode` and `editingStep` state machinery
  + the `isFreshDefault` heuristic — all simplified out by the
  no-accordion design (we just trust `billingMode` directly).
- `createWorkflowPR` PR body already links back to
  `pullfrog.com/console/<owner>/<repo>` with a "Verify workflow" CTA;
  no change needed there.

* fix(onboarding): provider tile labels — getProviderDisplayName expects slug

`getProviderDisplayName` from `pullfrog/internal` parses its argument as a
`provider/model` slug. Step 2 was passing bare provider keys (e.g.
"anthropic"), which made the helper throw "invalid model slug 'anthropic'
— expected 'provider/model'" and crashed the BYOK branch with the
page-level error boundary.

Replace with a local `providerDisplayName` that reads the registry
directly (`providers[key].displayName`). Drops the unused
`getProviderDisplayName` import.

Caught by Chrome DevTools end-to-end: clicking Bring-your-own-key on the
fresh wizard renders the page-level error. Re-verified post-fix: BYOK
flow shows step 2 with all 9 provider tiles correctly labeled
(Anthropic / OpenAI / Google / xAI / DeepSeek / Moonshot AI / Amazon
Bedrock / OpenRouter / OpenCode), step 3 reveals on tile click.

Also adds a guardrail to AGENTS.md: don't silently abandon visual
verification when DevTools breaks. Recovery is always possible
(pkill -9 chrome-devtools-mcp + pkill puppeteer + rm Singleton locks +
retry several times); if it genuinely won't recover, abort and tell the
user — never mask as "verified by code review".

* agents.md: never give up on Chrome DevTools MCP failures

Recovery is always possible (pkill chrome-devtools-mcp, remove Singleton
locks, retry several times). If genuinely unrecoverable, abort and tell
the user explicitly — never silently mask as "verified by code review".
Visual verification is non-negotiable for UI changes.

* onboarding: polish — checkbox color, redundant labels, copy

Caught during chrome-devtools verification of the BYOK + cross-page
selection flows:

- **Checkbox color**: native browser pink/red replaced with
  evergreen via `accent-evergreen-600`. Visually consistent with the
  rest of the wizard's selection states.
- **Bedrock provider tile**: was rendering "Amazon Bedrock" twice
  (provider name + recommended-model name both resolve to "Amazon
  Bedrock" because Bedrock has no `preferred` model under
  `providers.bedrock.models` — its single routing entry IS the
  recommended pick). Suppress the recommended subtitle when it
  duplicates the provider name.
- **Step 6 description**: tightened from a clunky two-clause sentence
  about workflow file landing to a single direct call: "Mention
  @pullfrog in any PR or issue to dispatch a run. (Branch-protected
  repos: merge the PR first.)"
- **Wizard intro**: was "Set up Pullfrog for your first repo" —
  outdated since multi-repo. Now: "Connect Pullfrog to your repos.
  Each step unlocks the next as you go."

Cross-page multi-select also verified: selections from page 1 persist
when navigating to page 2 and back. "N repos selected" counter
reflects total across all pages.

BYOK secret-add flow verified end-to-end: AddSecretModal opens with
the env var pre-filled, save triggers secrets refetch, step 3 flips
to "✓ ANTHROPIC_API_KEY configured", step 4 reveals automatically.

* onboarding: serial install, inline secrets, explicit credit redeem

- step 3: replace modal-based secret entry with inline password fields per
  provider, with deep links to provider dashboards. claude code OAuth
  surfaces as a distinct group when anthropic is picked. bedrock gets
  three-field form. github actions secrets path is collapsible with
  org/personal-aware urls + self-certify.
- step 4: merge repo-pick + workflow-create into one step. install is now
  serial (visible slow-reveal) instead of concurrent. continue button
  renders immediately on submit, disabled until every repo reaches a
  terminal state. errored rows render a single soft amber 'failed' label.
  pagination uses chevron buttons + keepPreviousData (no layout shift).
- step 6: explicit 'redeem $10 credit' for router+eligible, 'complete
  setup' otherwise. final redirect is a hard refresh so the repo grid
  picks up.
- signup credit: drop the mount-time auto-claim modal in favor of explicit
  user clicks. new useClaimSignupCredit hook + RedeemSignupCreditCallout
  banner inside RouterWalletSection so a BYOK→Router flip surfaces a
  one-click redeem affordance.
- billing mode is now optimistic (local state + background PATCH) and
  initialBillingMode + signupCreditEligible eager-load via server props
  to kill the multi-second click latency.
- skip onboarding: header button sets pullfrog_skip_onboarding cookie;
  server reads it in page.tsx and falls through to the regular grid.
- demo mode: NEXT_PUBLIC_ONBOARDING_DEMO=1 cycles the install progress
  list through pending/running/committed/PR/existing/failed states.
- createWorkflowForRepo: PULLFROG_FORCE_PR_CREATION=1 skips direct commit
  to exercise the PR fallback locally.

* onboarding: review feedback — focused eligibility query, best-effort model pre-fill, claim error toast

- billing/route.ts + console/[owner]/page.tsx: replace top-N
  recentGrants scan for signup-credit eligibility with a focused
  findFirst({ reason: { in: [SIGNUP, WELCOME] } }). the prior query
  could return any 5/10 rows (no orderBy on page.tsx) and miss a prior
  signup/welcome grant if a future grant reason (refund/referral/etc.)
  ever ships. recentGrants stays for the billing-history list.
- create-workflow/route.ts: gate Repo.model updateMany on result.type
  === "created" so an existing user-set model isn't clobbered when the
  workflow file already exists. wrap in try/catch: GitHub side effect
  already succeeded, so a transient DB blip shouldn't 500 the route
  and have the UI report failure on a partially-completed setup.
- SignupCreditModal: add onError toast to useClaimSignupCredit so
  transient redeem failures surface ("Couldn't redeem your credit. Try
  again in a moment."). callers .catch(() => null) the rejection so it
  doesn't propagate as an unhandled rejection in the React handler.
- OnboardingStepper: trim stale "per-row try again button" wording
  from progressRef + processRepo comments — that button was removed in
  the prior commit per design feedback.

* router: fix unspendable signup credit on no-card private repos (#791)

The bug
-------
`run-context/route.ts` gated `proxyModel` minting on `isInfraCovered`,
which is `oss || hasCard`. So a no-card account with positive wallet
balance (signup credit, top-up, etc.) on a private repo would never
get a `proxyModel` set on the run context. The action runtime then
fell through to whatever provider keys happened to be in the workflow
env — using the user's BYOK keys without their knowledge if any were
configured, or failing the run entirely otherwise.

Meanwhile `proxy-token/route.ts` already gated correctly on
`oss || hasCard || balance > 0`. The two routes disagreed, with
run-context being strictly more restrictive, so the agent never even
attempted to call proxy-token for these accounts. The wiki at
`billing.md:1052` documented the *intended* behavior ("a Router usage
row can debit a wallet with no card on file"), aspirational against
the actual code.

The action side had a parallel bug at `action/utils/proxy.ts:151` —
it re-derived `isInfraCovered({ isOss, plan })` and short-circuited
mint even when the server set `proxyModel`. Belt-and-suspenders that
was strictly more restrictive than the server.

Production impact
-----------------
Queried 55 router-mode no-card accounts holding signup credit:
- ALL have wallet balance = exactly $10.00 (untouched)
- ALL have 0 router proxy keys ever minted, 0 hwm usage
- ~25 have successful runs (using BYOK env vars from their workflow,
  unaware their credit isn't being touched)
- The rest have zero successes; some accumulated 25+ failures
  (e.g. `onechannelpe`: 25 failures, 0 successes, no card, $10 credit).

The fix
-------
- `run-context/route.ts`: widen `useRouter` to match proxy-token's
  gate. OSS short-circuits as before. Otherwise: router mode + card
  on file → mint; router mode + no card + positive balance → fetch
  balance, mint if > 0. Skip the balance read when a card is on file
  (auto-reload covers it without needing pre-flight balance — keeps
  the hot path single-query).
- `action/utils/proxy.ts`: drop the redundant `isInfraCovered` check.
  `ctx.proxyModel` IS the signal — the server is the authority on
  funding decisions; the action just trusts and mints.
- `wiki/pricing.md`: correct the Router proxy key minting gate row
  + add a paragraph explaining why this gate diverges from
  `isInfraCovered`.
- `wiki/billing.md`: rewrite the misleading "proxy-token returns 402"
  paragraph to describe what actually happens at both routes.

`isInfraCovered` is unchanged. It still gates Pullfrog-paid features
(learnings writes, indexing). The bug was in conflating "Pullfrog
pays for marginal infra" with "user can fund a Router run via wallet"
— different concerns, now untangled.

* revert: extract router-gate fix into its own PR

The router fix at a14bcdd4 is being shipped as a standalone PR so it
can be reviewed and merged independently of the onboarding-wizard
work. Reverting here keeps #762 focused on the wizard. The fix itself
landed at https://github.com/pullfrog/app/pull/792.

* router: fix unspendable signup credit on no-card private repos (#792)

* router: fix unspendable signup credit on no-card private repos (#791)

The bug
-------
`run-context/route.ts` gated `proxyModel` minting on `isInfraCovered`,
which is `oss || hasCard`. So a no-card account with positive wallet
balance (signup credit, top-up, etc.) on a private repo would never
get a `proxyModel` set on the run context. The action runtime then
fell through to whatever provider keys happened to be in the workflow
env — using the user's BYOK keys without their knowledge if any were
configured, or failing the run entirely otherwise.

Meanwhile `proxy-token/route.ts` already gated correctly on
`oss || hasCard || balance > 0`. The two routes disagreed, with
run-context being strictly more restrictive, so the agent never even
attempted to call proxy-token for these accounts. The wiki at
`billing.md:1052` documented the *intended* behavior ("a Router usage
row can debit a wallet with no card on file"), aspirational against
the actual code.

The action side had a parallel bug at `action/utils/proxy.ts:151` —
it re-derived `isInfraCovered({ isOss, plan })` and short-circuited
mint even when the server set `proxyModel`. Belt-and-suspenders that
was strictly more restrictive than the server.

Production impact
-----------------
Queried 55 router-mode no-card accounts holding signup credit:
- ALL have wallet balance = exactly $10.00 (untouched)
- ALL have 0 router proxy keys ever minted, 0 hwm usage
- ~25 have successful runs (using BYOK env vars from their workflow,
  unaware their credit isn't being touched)
- The rest have zero successes; some accumulated 25+ failures
  (e.g. `onechannelpe`: 25 failures, 0 successes, no card, $10 credit).

The fix
-------
- `run-context/route.ts`: widen `useRouter` to match proxy-token's
  gate. OSS short-circuits as before. Otherwise: router mode + card
  on file → mint; router mode + no card + positive balance → fetch
  balance, mint if > 0. Skip the balance read when a card is on file
  (auto-reload covers it without needing pre-flight balance — keeps
  the hot path single-query).
- `action/utils/proxy.ts`: drop the redundant `isInfraCovered` check.
  `ctx.proxyModel` IS the signal — the server is the authority on
  funding decisions; the action just trusts and mints.
- `wiki/pricing.md`: correct the Router proxy key minting gate row
  + add a paragraph explaining why this gate diverges from
  `isInfraCovered`.
- `wiki/billing.md`: rewrite the misleading "proxy-token returns 402"
  paragraph to describe what actually happens at both routes.

`isInfraCovered` is unchanged. It still gates Pullfrog-paid features
(learnings writes, indexing). The bug was in conflating "Pullfrog
pays for marginal infra" with "user can fund a Router run via wallet"
— different concerns, now untangled.

* action: drop dead isInfraCovered + plan param post-fix

Cleanup the action-side dead code introduced by the previous commit's
removal of the redundant `isInfraCovered` re-derivation in proxy.ts:

- delete `isInfraCovered` from action/utils/runContext.ts (was the only
  callsite; mirror in server's utils/billing.ts is unchanged and still
  load-bearing for learnings/indexing)
- drop unused `plan: AccountPlan` param from `resolveProxyModel` /
  `runProxyResolution` (and the corresponding `AccountPlan` import +
  the `plan: runContext.plan` arg at the main.ts call site)
- update the action/mcp/server.ts comment that pointed at the now-gone
  action mirror to reference the server-side `utils/billing.ts` instead

`AccountPlan` itself is still load-bearing (mcp/server, runContextData,
run-context fetch), only `isInfraCovered` and the dead `plan` parameter
go away.

* eager signup credit + free-OpenCode fallback when BYOK has no key (#789)

* eager signup credit + free-OpenCode fallback when BYOK has no key

addresses the silent-churn pattern that took out 15 first-run-failure
accounts post-launch: GH Actions secret references resolved to empty
strings (because the secrets didn't exist on the repo), the action
launched Claude Code with no key, the LLM provider 401'd, and the run
died in seconds with a synthetic "Invalid API key" message. those
accounts had no Router credits to fall back to because the lazy claim
required a dashboard visit they never made.

three changes, one PR:

1. Eager $10 signup credit at account creation. Both account-creation
   sites (`upsertAccountByClerkId` for dashboard signin, `fetchOrCreateRepo`
   for CLI / GH-App-only) now insert the `CreditGrant { reason: "signup" }`
   in the same transaction as the `accounts` row. CLI installers who
   never sign in get the credit. The dashboard `/signup-credit/claim`
   POST stays as an idempotent backstop for accounts created before
   this shipped.

2. Free-OpenCode fallback in the action. When the configured BYOK slug
   needs a provider key the runner doesn't have, swap to
   `opencode/minimax-m2.5-free` before agent selection so the run still
   succeeds. Surfaced via a `» fell back from <slug> to <free>` warning
   in the action log. Skipped on Router runs (Pullfrog mints the key)
   and when no model is configured (auto-select-with-throw still fires
   for the genuinely-misconfigured case).

3. New action-test fixture `byok-no-keys-fallback` that empty-strings
   every known provider key (matching how GH Actions handles missing
   secrets) and asserts the run succeeds with the fallback log line
   present. plus a unit test for the helper covering each skip case.

skipping the schema flip from `byok` to `router` — that's coming via
the onboarding-stepper PR (#762).

* fallback: skip Bedrock + surface in PR-comment footer

addresses copilot review on #789 (real bug — parseModel throws on
Bedrock raw IDs that have no slash, would crash before
validateBedrockSetup could surface its own error) and the user-side
ask to make the fallback visible in PR comments.

- selectFallbackModelIfNeeded skips when resolvedModel has no '/' so
  Bedrock routing IDs (e.g. us.anthropic.claude-opus-4-7) don't crash
  inside hasProviderKey -> parseModel. unit test covers it.
- toolState.modelFallback records the configured slug we fell back
  from. set in main.ts when fallback engages.
- buildPullfrogFooter accepts fallbackFrom and renders
  "Using `MiniMax M2.5` (free) (credentials for Claude Opus not
  configured)" so the substitution is visible in PR comments,
  reviews, PR bodies, and error reports.
- threaded through all four action-side footer call sites
  (mcp/comment, mcp/pr, mcp/review, utils/errorReport). server-side
  call sites in triggerWorkflow.ts / handleWorkflowRunWebhook.ts
  fire pre-action and don't have toolState — left as-is.

* fallback footer: use provider display name + document email asymmetry

addresses pullfrog reviewer findings on #789:

- footer now shows 'credentials for Anthropic not configured' (provider
  display name from `providers.anthropic.displayName`) instead of the
  per-model name. credentials are provider-scoped (ANTHROPIC_API_KEY
  covers all Anthropic models), so this matches what the user actually
  needs to fix.

- document the intentional asymmetry between eager and lazy signup
  credit paths: eager skips both the signupCreditClaimedEmail and the
  per-grant team@ alert. comment explains why (the 'new account
  created' alert already covers it on the eager path; the user-facing
  email assumes a user-initiated action that hasn't happened yet for
  CLI/GH-App-only signups).

- skipping the backfill for the 15 historical accounts per user's
  earlier decision — they all uninstalled, so the cohort self-selected
  out of being reachable.

* fallback: gate on resolvedModel + skip resolveModel re-resolve post-swap

local agnostic fixture run surfaced two real bugs the unit tests didn't
catch:

1. fallback gate was on configuredSlug (=payload.model) but the test
   uses PULLFROG_MODEL to set the model, which is read by resolveModel
   AFTER its slug arg. configuredSlug stayed undefined → fallback never
   fired. drop configuredSlug from the helper signature; gate purely on
   resolvedModel since that's the same value regardless of how the
   model was specified (DB config vs PULLFROG_MODEL env).

2. when fallback engaged, the post-swap resolveModel({slug: fallback.to})
   call was ALSO honoring PULLFROG_MODEL, re-overriding the fallback
   target back to the unkeyed model. validateAgentApiKey then threw
   "no API key found" against the original model. fix: skip the
   re-resolve. fallback.to is already a CLI-ready specifier.

unit tests updated for the new helper signature (8 tests, all pass).
fallback log line confirmed emitted in the local run pre-second-fix;
the second fix unblocks the validation that previously threw.

* models-bump: harden CI and bot prompt against catalog hallucinations

PR #790 (the first bot-authored models-bump PR) shipped a broken bump for
openrouter/gemini-flash: the bot pattern-matched the parallel google bump
and fabricated `openrouter/google/gemini-3.5-flash`, which exists on the
OpenRouter API but not on models.dev's openrouter section — the catalog
OpenCode actually reads. The slug failed at runtime with `Model not found`.
Two CI gaps let it through:

1. `models-live` matrix pruned every `openrouter/*` and keyed `opencode/*`
   alias as a "passthrough", smoke-testing only one canary per routing
   layer. But those aren't passthroughs — each is a distinct models.dev
   catalog entry that can drift independently of the direct-provider
   mirror. Drop the pruning; smoke every keyed alias (53 jobs, up from 25).
   Only `bedrock/byok` stays pruned (sentinel resolve).

2. `models-catalog` test (the integrity gate that asserts every resolve
   exists on models.dev) was main-only by design — to keep upstream catalog
   churn from blocking unrelated PRs. But it's exactly the test we want
   running on the bot's own catalog edits. Add `pullfrog/models-bump`
   head-ref to its trigger.

Also tighten the bot prompt in models-bump.yml: new rule 0 requires every
new `resolve` to equal `<alias-provider>/<c.modelId>` for some `c` in the
alias's own `candidates[]` in models-bump-context.json — the deterministic
preprocessor only emits candidates sourced from models.dev's mirror, so
this gates against the cross-alias pattern-matching that broke PR #790.
For `openRouterResolve` the gate is `openRouterCandidates[]` (OpenRouter
API), which is necessary but not sufficient; the `models-catalog` job is
the authoritative models.dev check.

Verified locally:
- baseline `pnpm -C action test:catalog` passes 133 tests
- simulated the PR #790 hunk (sed'd `openrouter/google/gemini-3.5-flash`
  into action/models.ts) and the catalog test fails with the right
  assertion: `model "google/gemini-3.5-flash" not found under openrouter
  on models.dev`
- `FULL=1 node action/test/matrix.ts` emits 53 aliases (was 25); every
  openrouter/* alias and every keyed opencode/* alias now smoked

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
2026-05-20 03:58:18 +00:00

171 lines
6.6 KiB
TypeScript

const MAX_STDERR_BYTES = 3000;
/**
* mutable per-run handle the agent harness writes to as a run progresses.
* the action's outer try/catch in `main.ts` reads this off `toolState` when
* the activity-timeout watchdog wins the race against the harness's own
* catch — the bare timer reject reason ("activity timeout: no output for
* 302s") tells the user nothing actionable, but `recentStderr` +
* `lastProviderError` together usually point straight at the upstream cause.
*
* `recentStderr` is shared by reference with the harness's bounded ring
* buffer, so the diagnostic always reflects the latest captured tail.
*/
export type AgentDiagnostic = {
/** display label for the agent, e.g. "Pullfrog". used in the headline. */
label: string;
/** shared reference to the harness's bounded stderr ring buffer. */
recentStderr: string[];
/** most-recent provider-error label from `detectProviderError`, if any. */
lastProviderError: string | undefined;
/** count of stdout events successfully parsed before the failure. */
eventCount: number;
};
/**
* Build a user-facing markdown body for an agent hang or failure.
*
* Rendered into both the PR progress comment and the GitHub Actions job
* summary. Returns `null` when no diagnostic is available, which signals to
* the caller to fall back to its bare-error rendering.
*
* `errorMessage` is the underlying timer / spawn reject string (e.g.
* `activity timeout: no output for 301s`). The idle seconds are parsed out
* of it for the hang explanation — total runtime would overstate the stall
* for runs that streamed for a long time before going quiet.
*/
export function formatAgentHangBody(input: {
diagnostic: AgentDiagnostic | undefined;
isHang: boolean;
errorMessage: string;
}): string | null {
if (!input.diagnostic) return null;
// billing exhaustion (CreditsError / FreeUsageLimitError / spending cap /
// Insufficient balance) is mis-classified as transient by upstream harnesses
// and the run only ends when the activity-timeout watchdog fires (see #778).
// when we recognise the billing label, replace the generic "stalled — auth
// error" headline with a billing-specific CTA that names the actual remedy.
if (input.diagnostic.lastProviderError === "provider billing exhausted") {
return formatBillingExhaustedBody(input.diagnostic);
}
const verb = input.isHang ? "stalled" : "failed";
const cause = input.diagnostic.lastProviderError
? ` — likely cause: \`${input.diagnostic.lastProviderError}\``
: "";
const headline = `**${input.diagnostic.label} ${verb}**${cause}`;
const explanation = formatExplanation({
isHang: input.isHang,
errorMessage: input.errorMessage,
});
const parts = [headline, "", `${explanation} ${formatEventsPart(input.diagnostic)}`];
const tail = renderStderrTail(input.diagnostic.recentStderr);
if (tail) {
// pick a fence longer than any backtick run in the body so a stderr line
// containing ``` (provider error JSON occasionally embeds it) can't
// terminate the fence early and corrupt the rest of the markdown.
const fence = pickFence(tail);
parts.push(
"",
"<details><summary>Recent agent stderr</summary>",
"",
fence,
tail,
fence,
"",
"</details>"
);
}
return parts.join("\n");
}
function formatExplanation(input: { isHang: boolean; errorMessage: string }): string {
if (!input.isHang) return `The agent exited unexpectedly: ${input.errorMessage}`;
const idleSec = parseIdleSec(input.errorMessage);
if (idleSec === undefined) {
return "The agent stopped emitting events and was killed by the activity-timeout watchdog.";
}
return `The agent stopped emitting events for ${idleSec}s and was killed by the activity-timeout watchdog.`;
}
function parseIdleSec(message: string): number | undefined {
const match = /no output for (\d+)s/.exec(message);
return match ? Number(match[1]) : undefined;
}
function formatEventsPart(diagnostic: AgentDiagnostic): string {
if (diagnostic.eventCount > 0) {
return `${diagnostic.eventCount} events were processed before the failure.`;
}
// when the provider-error label already names the cause in the headline,
// the reachability nudge below contradicts it (e.g. an immediate 401 also
// produces zero events but isn't a reachability problem). suppress it.
if (diagnostic.lastProviderError) return "No events were emitted before the failure.";
return "No events were emitted — check whether the model provider is reachable.";
}
function renderStderrTail(lines: readonly string[]): string {
if (lines.length === 0) return "";
const joined = lines.join("\n");
if (joined.length <= MAX_STDERR_BYTES) return joined;
return `... (older lines truncated)\n${joined.slice(-MAX_STDERR_BYTES)}`;
}
function pickFence(content: string): string {
let max = 0;
for (const match of content.matchAll(/`+/g)) {
if (match[0].length > max) max = match[0].length;
}
return "`".repeat(Math.max(3, max + 1));
}
/**
* Pull a billing URL out of the captured stderr if the provider helpfully
* embedded one (OpenCode Zen does — Anthropic and Gemini do not). Restricted
* to known billing/console hosts so a stray URL elsewhere in the buffer
* can't masquerade as the remedy link.
*/
function extractBillingUrl(lines: readonly string[]): string | undefined {
const urlPattern =
/https:\/\/(?:opencode\.ai\/[^\s"]*billing[^\s"]*|console\.anthropic\.com[^\s"]*|console\.cloud\.google\.com[^\s"]*billing[^\s"]*)/i;
for (let i = lines.length - 1; i >= 0; i--) {
const m = urlPattern.exec(lines[i] ?? "");
if (m) return m[0];
}
return undefined;
}
function formatBillingExhaustedBody(diagnostic: AgentDiagnostic): string {
const headline = `**${diagnostic.label} stopped** — your model provider returned a billing-exhausted response.`;
const billingUrl = extractBillingUrl(diagnostic.recentStderr);
const cta = billingUrl
? `Top up your provider balance, then re-run: [${billingUrl}](${billingUrl})`
: "Top up your model-provider balance (or rotate to a key with remaining credits) and re-run.";
const explanation =
"The agent kept retrying the request because the provider marked the failure as transient. Pullfrog's activity-timeout watchdog ended the run after no further events were emitted.";
const parts = [headline, "", explanation, "", cta];
const tail = renderStderrTail(diagnostic.recentStderr);
if (tail) {
const fence = pickFence(tail);
parts.push(
"",
"<details><summary>Recent agent stderr</summary>",
"",
fence,
tail,
fence,
"",
"</details>"
);
}
return parts.join("\n");
}