Files
shockbot/utils/providerErrors.ts
T
Colin McDonnell 5f3e46c42d fix: don't reuse disabled proxy key on workflow re-runs; non-fatal title-gen errors (#636)
* fix: don't reuse disabled proxy key on workflow re-runs; non-fatal title-gen errors

Three small surgical fixes addressing run https://github.com/pullfrog/app/actions/runs/25580969379:

1. **`/api/proxy-token` idempotency now checks `finalizedAt`.** GitHub re-runs
   share the same `run_id` (only `run_attempt` increments), so attempt N+1's
   action calls /api/proxy-token and inherits attempt N's `proxyKeyId`. The
   `workflow_run.completed` webhook between attempts retires that key on
   OpenRouter (`disableKey`), so attempt N+1 was getting back a disabled key
   and OpenRouter responded with `401 User not found` on every call. Falling
   through when finalized routes through the same billing gate
   (`handleRouterBilling` balance check), so no new attack surface.

2. **OpenCode title-gen / small-model errors no longer fatal.** OpenCode
   auto-spawns a small `agent=title small=true` background call at session
   start to name the thread, defaulting to `anthropic/claude-haiku-4.5`
   (anomalyco/opencode#1243). Pre-fix, the wrapper's `error` event handler
   treated any `type=error` as fatal, so a cosmetic title failure killed the
   run before primary inference even started. Now: stderr matching `small=true`
   sets a one-shot suppression flag for the next stdout `error` event, which
   is logged as a warning instead.

3. **Provider-error classifier puts auth patterns above rate-limit.** OpenRouter
   401 payloads bundle `x-ratelimit-*` response headers, and the loose
   `\brate[_ ]limit/i` pattern was winning. Added 401/403 status, `User not
   found`, `Invalid authentication`, `No auth credentials found` patterns
   ahead of rate-limit. Updated the existing 401-headers regression test to
   assert correct auth classification rather than `null`.

* opencode: correlate small-model error suppression by message, not by next-event

Pullfrog self-review on #636 flagged a real concurrency hole. OpenCode forks
the title-gen call (`session/prompt.ts:1452-1457` via `Effect.forkIn(scope)`)
so it races primary inference. The previous one-shot `suppressNextErrorEvent`
boolean had no per-call correlation: it was consumed by whichever stdout
`type=error` event landed next, regardless of which subagent produced it.
Under concurrent failures, a primary-agent error landing first could be
silently downgraded to a warning while the small-model error then propagated
fatally — the inverse of the bug the suppression was meant to prevent.

Replaced the boolean with a `Set<string>` of pending small-model error
messages. stderr extracts the inner `"message":"..."` from any classified
provider error tagged `small=true`; the stdout `error` handler suppresses
only when `event.error.data.message` matches a pending entry. Set is capped
at 32 entries so a long stream of small-model failures can't wedge memory.

Also corrected the comment that referenced "session summarizer" — verified
in opencode source that summarize() does NOT use `small: true`; only the
title generator does today (only `small: true` match in the codebase).

* revert: drop opencode title-gen suppression

We have no evidence — and can't construct a realistic scenario — where
title-gen fails on an otherwise-successful run. Title-gen and primary share
the same OPENROUTER_API_KEY and hit the same proxy/upstream; whatever breaks
one breaks the other. The original repro on run 25580969379 is fully
explained by the stale proxy key (fix #1) — title-gen happened to be the
first call that surfaced the auth error, but every subsequent primary call
would have died the same way.

Suppression code adds complexity (cross-stream correlation logic, message
matching, set capping) and a real failure mode of its own (a small-model
error with a unique message could mask an unrelated primary error landing
shortly after). Net negative. Removing.
2026-05-08 23:00:41 +00:00

76 lines
4.2 KiB
TypeScript

type ProviderErrorPattern = { regex: RegExp; label: string };
// status codes are only treated as provider errors when they are adjacent to
// a recognised status key. this rejects commit SHAs that happen to contain
// "429", version strings, file hashes, etc.
const statusKey = `\\b(?:status[_ ]?code|http[_ ]?status|status)["']?\\s*[:=]\\s*["']?`;
const PROVIDER_ERROR_PATTERNS: ProviderErrorPattern[] = [
// auth patterns must come BEFORE rate-limit patterns. OpenRouter 401 error
// payloads carry `x-ratelimit-*` response headers in the dump, and the
// free-form rate-limit regex below would otherwise win on word-boundary
// matches inside header names. canonical 401 messages: OpenRouter returns
// `{"error":{"message":"User not found","code":401}}` for disabled or
// invalid keys (https://openai.luzhipeng.com/docs/api/reference/errors-and-debugging).
{ regex: new RegExp(`${statusKey}401\\b`, "i"), label: "auth error (401)" },
{ regex: new RegExp(`${statusKey}403\\b`, "i"), label: "auth error (403)" },
{ regex: /\bUser not found\b/i, label: "auth error (invalid/disabled key)" },
{ regex: /\bInvalid authentication\b/i, label: "auth error (invalid credentials)" },
{ regex: /\bNo auth credentials found\b/i, label: "auth error (missing credentials)" },
{ regex: new RegExp(`${statusKey}429\\b`, "i"), label: "rate limited (429)" },
{ regex: new RegExp(`${statusKey}500\\b`, "i"), label: "provider 500 error" },
{ regex: new RegExp(`${statusKey}503\\b`, "i"), label: "provider unavailable (503)" },
// matches `rate limit`, `rate limited`, `rate limits exceeded`,
// `rate_limit_error`, `rate_limit_exceeded`. the leading `\b` + `[_ ]`
// separator rejects `x-ratelimit-*` / `anthropic-ratelimit-*` response
// headers (no separator between "rate" and "limit") which routinely
// appear in dumped 401 / 4xx error JSON.
{ regex: /\brate[_ ]limit/i, label: "rate limited" },
{ regex: /\bRESOURCE_EXHAUSTED\b/, label: "quota exhausted" },
// Google gRPC `INTERNAL` status. word-boundary anchors reject
// `INTERNAL_SERVER_ERROR` (HTTP 500 message that may appear in unrelated
// log lines) and identifiers like `INTERNALS`.
{ regex: /\bINTERNAL\b/, label: "provider internal error" },
{ regex: /\bUNAVAILABLE\b/, label: "provider unavailable" },
// matches `quota`, `insufficient_quota`, `quota_exceeded`, `quotaExceeded`.
// word-character lookarounds would reject `_quota` / `quotaX`; `quota` is
// specific enough that a plain substring match is safe.
{ regex: /quota/i, label: "quota error" },
// explicit zero-quota response, e.g. `{"limit": 0}`. the `\b` anchor
// around `limit` rejects keys like `time_limit` or `field_limit`.
{ regex: /["']?\blimit\b["']?\s*:\s*0\b/, label: "zero quota" },
];
export function detectProviderError(text: string): string | null {
for (const entry of PROVIDER_ERROR_PATTERNS) {
if (entry.regex.test(text)) return entry.label;
}
return null;
}
/**
* OpenRouter's response when the per-run key's remaining budget can't cover
* the agent's `max_tokens` reservation. Distinct from a generic provider error
* because it's a Pullfrog billing concern, not an upstream outage — the user's
* Router wallet ran out (or the key budget was undersized at mint time and the
* agent ran out of headroom partway through).
*
* Match must be specific to this exact OpenRouter error class. Generic "credits"
* or "limit" text shows up in unrelated errors and would mis-classify them.
*
* Sample:
* `APIError: This request requires more credits, or fewer max_tokens.
* You requested up to 32000 tokens, but can only afford 22800.`
*/
// `/s` (dotAll) lets `.*?` cross newlines so we still detect the error if any
// upstream layer reformats the message onto multiple lines. Without it, a
// single inserted `\n` would silently bypass the BillingError reclassification
// and the user would see the generic `❌ Pullfrog failed` dump instead of the
// actionable top-up CTA.
const ROUTER_KEYLIMIT_EXHAUSTED_PATTERN =
/requires more credits.*?fewer max_tokens|requested up to \d+ tokens.*?can only afford/is;
export function isRouterKeylimitExhaustedError(text: string): boolean {
return ROUTER_KEYLIMIT_EXHAUSTED_PATTERN.test(text);
}