6e94f513df
* feat(billing): monthly router spend limits (hard-cap + alert-only modes) (#660) Per-account ceiling on the sum of `router_topup` invoices (pending + succeeded) for the current UTC calendar month. Closes a gap where a runaway agent loop, leaked PR trigger, or stuck workflow could auto-reload indefinitely with no aggregate per-month ceiling. Two enforcement modes via `RouterLimitMode` enum: - `hard_cap`: refuse new auto-reloads; PR comment via reserveRun; 402 `router_monthly_limit` from /api/proxy-token; email + banner - `alert_only`: auto-reload keeps flowing; email + banner only, first breach per UTC month Enforcement is split across reserveRun (pre-dispatch paywall comment) and /api/proxy-token phase-1 (mid-run 402). Both surfaces read through the same `getRouterSpentThisMonthCents` helper so the dashboard, the dispatch gate, and the auto-reload gate can't disagree. Email dedup uses `Account.routerLimitNotifiedMonth` (YYYY-MM string), claimed atomically inside the phase-1 SERIALIZABLE txn so concurrent reloads breaching together send exactly one email. Read-time comparison with the current month re-arms on rollover — no cron. Admin surface: `RouterLimitBanner` (reuses `DelinquencyBanner` shell) above the Router/BYOK tabs in `ModelAccessCard`, with a popover "Adjust limit" form that PATCHes the existing /api/account/[owner]/billing/settings route. Same `assertBillingAdmin` gate that owns the other billing settings — no new auth surface. See wiki/billing.md § Router monthly spend limit for the full contract + edge cases. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): anneal pass on monthly Router spend limit (#660) Round-1 review across 5 lenses (billing-subsystem, correctness, security, operational-readiness, research-validated-assumptions) surfaced one critical + three actionable major findings on top of [#748](https://github.com/pullfrog/app/pull/748). **Critical — CAS never matched NULL.** `claimRouterLimitNotificationSlot` used Prisma `NOT { routerLimitNotifiedMonth: monthKey }`, which compiles to `field != value` — UNKNOWN (not TRUE) against the post-migration `NULL` default. First breach for any account would never claim the slot, never stamp the row, and never fire the email (hard_cap or alert_only). Replaced with `OR: [{ field: null }, { field: { not: monthKey } }]`, mirroring the `maybeNotifyLowBalance` pattern. **Major — email gap on manual-top-up over cap.** Breach email was only wired through `/api/proxy-token`. A manual `/billing-top-up/<owner>` that crosses the cap blocks dispatch via `reserveRun` but never hits proxy-token, so the user got the PR comment but no email. Wired the CAS + `after(maybeNotifyRouterLimit)` into `reserveRun`'s PaywallError catch (the SERIALIZABLE txn rolled back when we threw, so we re-claim with the global client; single-statement CAS is its own race boundary against concurrent proxy-token claims). **Major — PR paywall comment leaked $ figures.** `router_limit` body embedded `($X of $Y)` in a comment visible to anyone with PR read access (public repos, forks, outside collaborators). Other paywall types deliberately avoid amounts. Removed; deep link still points to the authenticated console for the figures. **Medium — observability.** Added `[router-limit]` structured logs at the three enforcement sites (proxy-token hard_cap 402, proxy-token alert_only breach, reserveRun paywall) so on-call can grep "did the cap fire for customer X this month." **Medium — customer docs.** Added a `### Monthly spend limit` section to `docs/billing.mdx` (Mintlify) describing the two modes and the manual-top-up caveat. **Doc — refund/dispute interaction.** Documented in `wiki/billing.md` that the cap inherits the existing webhook semantics: disputed `router_topup` drops from the sum (cap briefly un-trips); refunds don't flip status today so refunded top-ups keep counting. Matches wallet behavior — not redefined here. Accepted as-is (documented or pre-existing): `after()` reliability vs stamp-before-send tradeoff, alert_only email fires before Stripe phase-2, proxy-token reads limit fields outside SERIALIZABLE scope (brief TOCTOU on admin lowering cap), stale paywall comment on cap clear, no global kill switch (per-account `alert_only` flip is the practical kill switch), no audit log on cap changes (no existing audit infra), action version not bumped (separate release commit). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): anneal round 2 on monthly Router spend limit Round-2 anneal (billing-subsystem, correctness, research-validated, user-journey, operational-readiness) surfaced a critical merge conflict and a handful of major correctness + UX gaps on top of [#748](https://github.com/pullfrog/app/pull/748). **Critical — merge conflict.** While #748 was open, [#755](https://github.com/pullfrog/app/pull/755) extracted `formatBillingErrorSummary` from `action/main.ts` to `action/utils/billingErrors.ts`. The PR's new `router_monthly_limit` arm still lived in `action/main.ts`. Took main's slim orchestrator wholesale; moved the arm into the extracted file. **Major — cap = payments only, not dispatch.** `reserveRun` was pre-empting all PR-comment / `/trigger` dispatch on `spent >= limit` regardless of wallet balance, contradicting the cap's positioning as "ceiling on what you pay." An account with $500 of paid-up wallet and a breached $100 cap couldn't trigger any new run via the comment path, while GitHub UI re-runs (which bypass `reserveRun`) succeeded — surface inconsistency. Deleted the pre-dispatch gate; `/api/proxy-token` is now the sole enforcement point, refusing only the next auto-reload that would push past. Wallet credit always drains. Dropped the now-dead `router_limit` arm in `buildPaywallCommentBody`, the dead `routerSpentCents`/`routerLimitCents` fields on `PaywallError.detail`, and the post-paywall email-fire David added — all unreachable. **Major — split `manual_topup` from `router_topup`.** Manual on-session top-ups at `/billing-top-up/<owner>` were landing as `Invoice.kind = "router_topup"` and counting toward the cap. The cap exists to brake *passive* runaway (auto-reload loops); a manual top-up is a deliberate click-through that the user owns. Added `InvoiceKind.manual_topup`, flipped the manual write site + `createTopUpCheckoutSession` metadata, broadened wallet / reconcile / billing-report reads to `kind IN (router_topup, manual_topup)`, and scoped `getRouterSpentThisMonthCents` (the cap aggregate) to `router_topup` only. Worked example: cap=$300, reload=$100 → exactly three reloads succeed; a fourth is blocked. Historical rows stay labelled `router_topup` (no backfill); the asymmetry is small and accepted since the manual flow only existed alongside auto-reload for a brief window. Extended the `invoices_kind_matches_stripe_columns` CHECK so `manual_topup` follows the same shape as `router_topup` (PaymentIntent-backed, no stripeInvoiceId); split into a second migration because PG forbids using a freshly-added enum value in the same transaction. **Major — email reframed around the triggering reload event.** The `alert_only` body was reporting a pre-eager-write `spentCents` while the dashboard reads the post-commit value, so email and dashboard disagreed by exactly one reload. Both flavors now say "Your most recent $50 auto-reload brought you over your $300 monthly limit" instead of a running spent-of-cap total — no reconciliation needed, no more "you've hit your monthly cap" copy firing for partial breaches (spent=$80 of $100, reload=$30 would have triggered that wording). **Major — `/trigger/<owner>/<repo>/<n>` paywall copy.** Hardcoded "You've used your 30 free runs this month. Add a card to continue at 7¢/run." regardless of `detail.reason`. Branched on `cap` vs `delinquent` so each paywall surfaces actionable copy with the right CTA. `router_limit` no longer flows through here (per F4 above). **Major — RouterLimitBanner.** Added an `isAlertBreached` visual state (amber palette) so an `alert_only` account at $240 of $200 no longer renders in the same neutral zinc chrome as a healthy under-cap account. Updated popover copy to reflect the auto-reload-only scope. **Medium — paywall log line.** Added `detail.reason` to the `[Installation X] paywall:` log so on-call grepping for "why was this paused" can distinguish `cap` from `delinquent`. **Cleanup.** Dropped dead `utcMonthKey` import + re-export in `maybeNotifyRouterLimit.ts`. Renamed file-internal `reconcileRouterTopup*` fns + their reconcile-kind labels to `reconcileTopup*` / `topup_*` since they now handle both kinds. Updated wiki/billing.md + docs/billing.mdx + schema doc comments throughout. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(billing): drop routerLimitNotifiedMonth sentinel; rely on Resend idempotency-key The sentinel was the same anti-pattern as `routerLowBalanceEmailedAt` sitting next to it — a single-purpose state column on `Account` that encoded a date as a string and required a custom CAS predicate to read/write race-safely. Plus it had real holes: Resend send failure left the sentinel stamped and the account silently un-emailed for the month (F11), mode flips mid-month didn't re-arm (F9), and cap-lowered edge cases never fired at all. Replace it with: fire `maybeNotifyRouterLimit` on every breaching reload, let the Resend `Idempotency-Key` `router_limit:<accountId>:<monthKey>:<mode>` collapse repeats inside Resend's 24h dedup window. Continuously-breaching accounts get ~1 reminder per day; brief Resend outages self-heal because the next breaching reload re-attempts the send. Mode is in the dedup key so `alert_only → hard_cap` mid-month re-arms a fresh email with the appropriate copy. Drops `Account.routerLimitNotifiedMonth` and `claimRouterLimitNotificationSlot`; simplifies the proxy-token phase-1 branch significantly. Net diff is negative LOC and the data model loses a single-purpose sentinel. Migration was branch-local — never deployed — so I edited the original add-cap migration in place to drop the column from the ALTER TABLE rather than chain a drop-column migration on top. Preview Neon branches reset automatically on history rewrite per wiki/migrations.md. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): hide RouterLimitBanner when no cap is configured The banner was unconditionally rendered for every billing-enabled account, including pure-BYOK admins who never touch Router. They got "No monthly spend limit / Router has spent $0.00" + a divider as visual noise on the model access page — basically nagging them to set a feature they may not want. Running without a cap is valid; we don't nag. ModelAccessCard now gates the banner block (banner + dividers) on `routerMonthlyLimitCents !== null`. RouterLimitBanner drops the no-limit visual state, the "Set monthly limit" CTA text, and the dead `hasLimit` branching. Cleaner three-state shape (under cap / amber breached / brick breached). Discoverability: no-cap users no longer see a UI affordance to set one. That's deliberate — the cap is a power-user feature documented in docs/billing.mdx. If discoverability becomes an ask, we can add a small inline link inside RouterWalletSection without bringing back the always-visible banner. Resolves the only outstanding finding from cursor bugbot's review of ff5328c (banner-visible-for-byok thread). Co-authored-by: Cursor <cursoragent@cursor.com> * docs(billing): docs/wiki match new "no banner without a cap" reality Pullfrog bot review of f7672ca pointed out the customer docs still told users to "Set the cap from the **Monthly spend limit** banner in the **Model costs** card" — but after hiding the banner for no-cap accounts there is no such banner to use until you already have a cap. Catch-22 for first-time setup. Rewrote docs/billing.mdx to be self-contained: explain what the cap is, what the two modes do, what the banner shows *once configured*, and direct admins to PATCH the billing settings endpoint (or reach out to support) for first-time setup. Cap is positioned as optional; running without one is the documented default. Wiki paragraph in wiki/billing.md updated to match — banner is only rendered when a cap exists, three visual states (under / amber / red), no first-time-setup UI nag by design. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): move monthly cap into RouterWalletSection as a normal settings row; drop the banner entirely The standalone `RouterLimitBanner` was the wrong shape. It only rendered when a cap was already configured (so there was no UI to discover the feature in the first place — first-time setup required hitting the API directly), and it occupied prominent real estate above the tabs to surface state that already lives in the row's own input when the form moves down where it belongs. New shape: monthly cap is just a third row inside `RouterWalletSection` sibling to **Auto-reload amount** and **Auto-reload threshold**. Gated the same way (card on file + auto-reload enabled — the only state where the cap actually means anything). Empty input → no cap, with placeholder "No limit". Setting a number reveals a **Behavior at limit** toggle built on the same `Tabs` slider component used for the Router/BYOK tab switch, so the look matches the rest of the card. Deletes: - `RouterLimitBanner` component (212 lines) - banner mount + conditional + spacers in `ModelAccessCard` - `AlertTriangle` is still imported (used by `DelinquencyBanner`) Adds: - one settings row in `RouterWalletSection` with the cap input + mode tabs - `routerMonthlyLimitUsd` / `routerLimitMode` plumbed through the existing `saveSettings` helper (widened to accept `string | null`) - `Tabs` / `TabsList` / `TabsTrigger` import Docs + wiki updated to match the new shape; the customer doc no longer points at a banner that won't appear. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): split monthly cap input and Behavior-at-limit toggle into separate rows with hr between Previously bundled both into one row block. Restructure: cap input is its own row; Behavior-at-limit Tabs gets a sibling row with the standard `h-5 + hr + h-5` separator between (matching the rhythm of auto-reload amount → threshold → monthly cap). Mode-toggle row is gated on `routerMonthlyLimitCents !== null` so the hr + tabs only appear once a number is in the cap input. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(billing): right-justify Behavior-at-limit Tabs to mirror Auto-reload toggle row Same `flex items-center justify-between gap-3` layout as the Auto-reload row: label group on the left, control on the right. Drops the vertical stack in favour of the horizontal one — looks identical to the toggle row directly above. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
190 lines
8.0 KiB
TypeScript
190 lines
8.0 KiB
TypeScript
/**
|
|
* Billing-error classification + user-facing copy for `/api/proxy-token`
|
|
* failures and OpenRouter mid-run exhaustion. Two error classes (Billing vs.
|
|
* Transient) keep the framing honest: a card decline is *not* the same UX as
|
|
* a 503 from the proxy service. Both originate in `utils/proxy.ts` (mint
|
|
* failures) and `utils/runErrorRenderer.ts` (mid-run keylimit reclassify).
|
|
*
|
|
* Renderers return markdown bodies that are written into both the GitHub
|
|
* Actions job summary and the PR progress comment.
|
|
*
|
|
* Lives outside `main.ts` so adding a new error `code` branch is a one-file
|
|
* edit that does not retrigger the full LLM CI matrix (`action/main.ts` is
|
|
* in `action/test/coverage.ts::ALWAYS_RUN_ALL`).
|
|
*/
|
|
|
|
/**
|
|
* Billing-layer error surfaced from `/api/proxy-token` as a 402. User-actionable
|
|
* — distinct from TransientError (503 / transient sync issue) so the job
|
|
* summary + PR comment can use affirmative "you need to do X" copy rather than
|
|
* the ambiguous "billing error" label that makes transient outages look like
|
|
* the user's fault.
|
|
*
|
|
* `code` is a server-side discriminator: `router_requires_card` (no card + no
|
|
* wallet balance on Router), or null for unclassified. `declineCode` is
|
|
* Stripe's more specific sub-reason on `card_declined` (e.g.
|
|
* `insufficient_funds`, `lost_card`). `needsReauthentication` is the 3DS case
|
|
* broken out for convenience.
|
|
*/
|
|
export class BillingError extends Error {
|
|
code: string | null;
|
|
declineCode: string | null;
|
|
needsReauthentication: boolean;
|
|
|
|
constructor(
|
|
message: string,
|
|
opts: {
|
|
code?: string | null;
|
|
declineCode?: string | null;
|
|
needsReauthentication?: boolean;
|
|
} = {}
|
|
) {
|
|
super(message);
|
|
this.name = "BillingError";
|
|
this.code = opts.code ?? null;
|
|
this.declineCode = opts.declineCode ?? null;
|
|
this.needsReauthentication = opts.needsReauthentication ?? false;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Transient service failures from `/api/proxy-token` (503: partial OpenRouter
|
|
* usage sync, DB flake, in-flight payment intent). Not the user's fault — the
|
|
* summary uses "temporarily unavailable" framing, and the non-zero exit lets
|
|
* GH Actions apply whatever retry policy the workflow has configured.
|
|
*/
|
|
export class TransientError extends Error {
|
|
constructor(message: string) {
|
|
super(message);
|
|
this.name = "TransientError";
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Deep link into the right console section for the failing account. Anchors
|
|
* are defined in `app/console/[owner]/page.tsx` (`#billing`, `#model-access`).
|
|
* `owner` is the GitHub login of the repo's account — i.e. the org or user
|
|
* that pays for this repo's runs, which is the right scope for billing.
|
|
*/
|
|
function billingConsoleUrl(owner: string, anchor: "billing" | "model-access"): string {
|
|
return `https://pullfrog.com/console/${encodeURIComponent(owner)}#${anchor}`;
|
|
}
|
|
|
|
/**
|
|
* Render a BillingError as user-facing markdown (shared between GH job summary
|
|
* and the PR progress comment). Goals:
|
|
*
|
|
* - quiet, not alarmist — bold first line instead of an `### ❌` H3, since
|
|
* the comment already has Pullfrog branding in the footer
|
|
* - actionable — every branch ends in a single CTA deep-linked to the
|
|
* correct section of the owner's console
|
|
* - honest — say what actually went wrong (card declined vs. balance
|
|
* empty vs. 3DS required), don't lump them under "billing error"
|
|
*
|
|
* Branches:
|
|
* - `router_requires_card`: user is on Router mode with no card AND no
|
|
* wallet balance (signup credit exhausted or not granted). Frame as
|
|
* "add a card to continue", link to `#model-access` where the Add
|
|
* Card flow lives.
|
|
* - `router_balance_exhausted`: user has a card on file but auto-reload is
|
|
* disabled and they've spent past their $5 overdraft buffer. Frame as
|
|
* "balance ran out" and surface both remediation paths (top up, or flip
|
|
* on auto-reload).
|
|
* - `router_keylimit_exhausted`: OpenRouter rejected mid-run because the
|
|
* per-run key budget was exhausted while the agent was working. The
|
|
* wallet is now negative; same remediation as `router_balance_exhausted`
|
|
* but framed for the after-the-fact case ("this run was cut short").
|
|
* - `needsReauthentication`: issuer requires 3DS on every off-session
|
|
* charge. Re-adding the card won't help — the only escape is a manual
|
|
* top-up where 3DS runs interactively in Stripe Checkout.
|
|
* - `declineCode` set: Stripe declined a real charge. Show the sub-code
|
|
* so support can act on it; tell the user we'll retry on next dispatch.
|
|
* - default: balance hit zero with no in-flight charge (auto-reload off
|
|
* or amount below threshold). Direct them to top up or enable auto-reload.
|
|
*/
|
|
export function formatBillingErrorSummary(error: BillingError, owner: string): string {
|
|
if (error.code === "router_requires_card") {
|
|
return [
|
|
"**Add a card to start using Pullfrog Router.**",
|
|
"",
|
|
"Router proxies OpenRouter at raw cost — no platform markup. Add a card and we'll auto-reload your wallet so runs keep flowing.",
|
|
"",
|
|
`[Add a card →](${billingConsoleUrl(owner, "model-access")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
if (error.code === "router_balance_exhausted") {
|
|
return [
|
|
"**Your Pullfrog Router balance is exhausted.**",
|
|
"",
|
|
"You have a card on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.",
|
|
"",
|
|
`[Top up balance →](${billingConsoleUrl(owner, "billing")}) · [Enable auto-reload →](${billingConsoleUrl(owner, "model-access")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
if (error.code === "router_keylimit_exhausted") {
|
|
return [
|
|
"**This run was cut short — your Pullfrog Router balance ran out mid-run.**",
|
|
"",
|
|
"OpenRouter stopped the agent because the per-run budget was exhausted. Your wallet is now negative; top up or enable auto-reload to keep runs flowing.",
|
|
"",
|
|
`[Top up balance →](${billingConsoleUrl(owner, "billing")}) · [Enable auto-reload →](${billingConsoleUrl(owner, "model-access")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
if (error.code === "router_monthly_limit") {
|
|
return [
|
|
"**Pullfrog Router hit its monthly spend limit.**",
|
|
"",
|
|
"Auto-reloads are paused for the rest of this UTC month. Ask your admin to raise the cap, or wait for it to reset at 00:00 UTC on the 1st.",
|
|
"",
|
|
`[Adjust limit →](${billingConsoleUrl(owner, "model-access")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
if (error.needsReauthentication) {
|
|
const code = error.declineCode ?? "authentication_required";
|
|
return [
|
|
`**Your card issuer requires 3D Secure on every charge** (\`${code}\`).`,
|
|
"",
|
|
"Pullfrog can't complete a 3DS challenge from inside a workflow. Top up your Router balance once in Stripe Checkout — subsequent runs draw from the prepaid balance without re-triggering 3DS.",
|
|
"",
|
|
`[Top up balance →](${billingConsoleUrl(owner, "billing")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
if (error.declineCode) {
|
|
return [
|
|
`**Your card was declined** (\`${error.declineCode}\`).`,
|
|
"",
|
|
"Update your payment method and Pullfrog will retry on the next run.",
|
|
"",
|
|
`[Update payment method →](${billingConsoleUrl(owner, "billing")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
return [
|
|
"**Your Pullfrog balance is empty.**",
|
|
"",
|
|
"Top up your balance or enable auto-reload to keep runs flowing.",
|
|
"",
|
|
`[Manage billing →](${billingConsoleUrl(owner, "billing")})`,
|
|
].join("\n");
|
|
}
|
|
|
|
/**
|
|
* Render a TransientError as user-facing markdown. Distinct framing from
|
|
* BillingError so the user doesn't read an alarm and assume their card
|
|
* failed — this branch is "our fault, retry shortly", not theirs.
|
|
*/
|
|
export function formatTransientErrorSummary(error: TransientError, owner: string): string {
|
|
return [
|
|
"**Pullfrog billing is temporarily unavailable.**",
|
|
"",
|
|
error.message,
|
|
"",
|
|
`Usually transient — the next dispatch should succeed. If it persists, check [status.pullfrog.com](https://status.pullfrog.com) or [your console](${billingConsoleUrl(owner, "billing")}).`,
|
|
].join("\n");
|
|
}
|