fe2746198c
* fix: 6 unaddressed log-audit / run-audit findings - #836 + #818 (clerk middleware SyntaxError on action-runtime endpoints): narrow proxy.ts matcher to exclude /api/repo/<owner>/<repo>/run-context, /api/runtime/*, /api/proxy-token. these are server-to-server with their own auth and have no Clerk session to evaluate, so clerkMiddleware's decodeJwt throws turn into 500s on every request. - #837 (npx EBADDEVENGINES on customer's package.json): change runCli's bootstrap cwd from $GITHUB_WORKSPACE to os.tmpdir() so npm v11+ doesn't enforce devEngines.packageManager from the customer's tree before our bootstrap can install pullfrog. CLI process.chdir's to payload.cwd internally, so the runtime work still happens in $GITHUB_WORKSPACE. - #838 (createWorkflowDispatch silently dropped 39 user runs on a 5xx spike): add bounded retry (3 attempts, ~750ms total) on Octokit 5xx and network errors inside dispatchReservedRun. preserves the existing 422 "Unexpected inputs" path. retry budget stays well under GitHub's 10s webhook redelivery window. - #833 (bail() redirect("/signout") propagated NEXT_REDIRECT into webhook handlers, 40+ 500s/24h): drop the redirect side-effect; bail now just classifies bad-credentials as non-retryable and propagates. UI flows that wanted auto-signout on revoked tokens can detect it themselves; the side-effect was wrong for any non-page caller. - #835 (BYOK provider billing-exhausted fell through to raw error renderer, 37 review-mode runs/24h with no PR-side signal): - extend providerErrors patterns with Anthropic "credit balance is too low" + extract isProviderBillingExhausted / extractProviderId helpers - add a renderer branch in runErrorRenderer.ts that names the provider (parsed from providerID=) and links to its billing dashboard - route handleAgentResult's !result.success path through the renderer + reportErrorToComment with createIfMissing, so review-mode and silent triggers get an actionable PR comment regardless of mode - #834 (post-hook ERR_MODULE_NOT_FOUND already fixed on main, hardening): - add a vitest invariant that walks the entryPost.ts import graph and refuses any non-relative / non-node: specifier — catches the next `@actions/core` slip-up before publish - add an analyze-logs classifier so future entryPost crashes surface as failure:post-hook-module-not-found instead of hiding inside failure:unknown / failure:git-lock-file Co-authored-by: Cursor <cursoragent@cursor.com> * anneal: fix dead-code matcher, 404 url, silent-trigger gate, and grammar regression Round-1 anneal pass on the audit-fixes PR surfaced two critical issues + several majors that the original fixes shipped with: - proxy.ts matcher 1 (`(?!_next|[^?]*\.(...))`) still caught every /api/... route because matcher arrays are OR'd. The narrowing in matcher 2 was dead code → middleware still 500'd on /api/runtime/*, /api/proxy-token, /api/repo/.../run-context. Carve-out now lives in BOTH matchers. - opencode.ai/billing returns 404; canonical top-up surface is /zen. deepseek /usage is the consumption page, not the top-up flow (/top_up is correct). google /apikey is the keys list, not billing (/usage is the spend dashboard). All three URL strings updated. - handleAgentResult gated reportErrorToComment behind `if (!ctx.silent)`, contradicting the createIfMissing intent — silent IncrementalReview / pull_request_synchronize / auto-label still got zero PR signal on BYOK billing exhaustion, the exact failure mode #835 was meant to fix. Moved createIfMissing into finalizeSuccessRun's existing render-and- post block (single source of truth), reverted handleAgentResult to its prior shape. Side benefit: drops the double-PATCH that fired on every non-silent !success path with an existing progress comment. - Anthropic-direct error rendered "**Your your provider account is out of credit.**" because Anthropic SDK has no providerID= tag, so extractProviderId returned null and the headline composed "Your " + "your provider". Added detectProviderId Anthropic fallback (matches "Anthropic API" / "credit balance is too low") so the link is reachable AND made the headline conditional on whether a provider id was detected. - Reordered renderRunError classifier: BYOK billing-exhausted now runs BEFORE api-key auth detection. Providers commonly return 401 for billing exhaustion (DeepSeek, Gemini), and the OpenCode harness logs often include "API Error: 401" in the raw error body, which isApiKeyAuthError would otherwise match — surfacing "rotate your key" when the actual fix is "top up credits". - isTransientUpstreamError missed ENOTFOUND / ENETUNREACH / EHOSTUNREACH (undici DNS-class failures Octokit doesn't wrap with a status). Added to the prefix alternation. - Tightened "10s webhook redelivery budget" / "GitHub redelivers" wording in triggerWorkflow.ts and bail.ts JSDoc — GitHub's 10s is the response timeout (it doesn't auto-redeliver); upstream webhook proxy retries are what multiplied the failure. Co-authored-by: Cursor <cursoragent@cursor.com> * anneal r2: unbreak proxy.ts matcher 2, extend carve-out, mkdtemp bootstrap Round-2 anneal (security + cross-cutting lenses) on top of the round-1 audit-fixes commit caught a critical regression + several majors: - proxy.ts matcher 2 from r1 (`/(api|trpc)(?!...)(.*)`) does NOT compile. path-to-regexp rejects a top-level `(?!` after `)` as "Pattern cannot start with '?'", and Next.js's SourceSchema runs the same validator at build time and aborts via `process.exit(1)`. PR #840's Vercel + preview deployments have been failing since 978eca26 for exactly this reason. Fixed by nesting the lookahead inside an outer parameter group: `/(api|trpc)((?!...).*)`. Same shape matcher 1 already uses, which is why m1 always compiled. Verified `pnpm next build` succeeds end-to-end. - proxy.ts carve-out was incomplete relative to its own justification. The "no Clerk session, decodeJwt 500" failure mode applies to ALL server-to-server action-runtime endpoints — five more share the exact shape: /api/repo/.../learnings, /api/repo/.../pr/.../summary-comment, /api/repo/.../issue/.../plan-comment, /api/workflow-run/, and /api/github/installation-token. Extended both matchers. /api/upload/ signed-url stays in (dual auth: Clerk session OR bearer JWT — needs middleware for the user path). - proxy.ts carve-outs were unanchored: a future /api/proxy-token-info, /api/proxy-tokens, or /api/repo/X/Y/run-context-foo would silently bypass Clerk. Added `(?:$|/)` for path-prefix carve-outs (allow exact match or sub-path), `$` for routes with browser-callable siblings (e.g. `learnings/history` is browser-Clerk, `learnings$` is action- bearer-JWT). Verified 30/31 routes via path-to-regexp test harness. - runCli.ts cwd flipped from $GITHUB_WORKSPACE to os.tmpdir() in r1 (#837 fix for npm v11 devEngines.packageManager EBADDEVENGINES). But $TMPDIR is overridable from a prior $GITHUB_ENV step — a customer- authored or compromised prior step can plant /atk/node_modules/ pullfrog/ and `echo "TMPDIR=/atk" >> $GITHUB_ENV`, and our npx --yes pullfrog@<v> bootstrap resolves the local install first, executing attacker code with full action env (provider keys, OIDC, installation token, CODEX_AUTH_JSON). Switched to mkdtempSync(join (tmpdir(), "pullfrog-bootstrap-")) — fresh per-invocation 0700 dir, not pre-writable by anything earlier in the job. - runLifecycle.ts: writeRunErrorOutputs (catch-path) didn't pass createIfMissing: true, contradicting the symmetric intent of the r1 finalizeSuccessRun fix. Silent triggers (IncrementalReview / pull_request_synchronize / auto-label) that throw past the success path still got zero PR signal — exact failure mode #835 was meant to close. Now both paths pass createIfMissing: true. - runErrorRenderer.ts detectProviderId regex `/Anthropic API|credit balance is too low/i` could mis-tag a non-Anthropic billing-exhausted error that mentioned "Anthropic API" in passing (fallback-chain agent prompt text, OpenCode harness logs). Tightened to /credit balance is too low/i — Anthropic-specific phrasing, sufficient for the direct- Anthropic SDK case the fallback exists to handle. - runErrorRenderer.ts JSDoc: r1's classifier reorder put hang at #6 in code but the JSDoc still listed it at #2. Reordered the doc to match dispatch order, with explicit note that hang is a sub-source for the api-key check (which is why hangBody is precomputed early). - analyze-logs.ts: ERR_MODULE_NOT_FOUND.*entryPost regex needs `s` flag so it survives Node v23+ stack-trace reformatting onto multiple lines. One-char fix to defend the #834 classification bucket. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
212 lines
8.3 KiB
TypeScript
212 lines
8.3 KiB
TypeScript
/**
|
|
* Classify + render the error thrown out of the main run try-block into a
|
|
* pair of user-facing markdown bodies — one for the GitHub Actions job
|
|
* summary tab, one for the PR progress comment.
|
|
*
|
|
* Classifications, in dispatch order (first match wins; the api-key
|
|
* branch additionally folds in the activity-timeout hang body as a
|
|
* sub-source so a hang masking an api-key error still surfaces the api-key
|
|
* CTA):
|
|
*
|
|
* 1. `BillingError` — either the proxy-token mint already threw one (402
|
|
* handled inline) or the agent runtime surfaced an OpenRouter
|
|
* "key budget exhausted" string mid-run. Both render via
|
|
* `formatBillingErrorSummary` so the user sees actionable copy.
|
|
*
|
|
* 2. BYOK provider billing-exhausted (#835) — DeepSeek "Insufficient
|
|
* Balance", Anthropic "credit balance is too low", OpenCode Zen
|
|
* `CreditsError`, Gemini "spending cap". Checked before api-key auth
|
|
* because billing-exhausted responses often carry 401 status codes
|
|
* that `isApiKeyAuthError` would otherwise mis-classify.
|
|
*
|
|
* 3. API-key auth error — `isApiKeyAuthError` sniffs the raw error string
|
|
* (or the activity-timeout hang body when present, since that's where
|
|
* the underlying provider error often lands); `formatApiKeyErrorSummary`
|
|
* renders provider + console-link copy.
|
|
*
|
|
* 4. ProviderModelNotFoundError — stale free-fallback model id no longer
|
|
* in the OpenCode catalog; renders a nudge to add a BYOK key.
|
|
*
|
|
* 5. Activity-timeout hang — `errorMessage` starts with
|
|
* `"activity timeout"` or `"agent still pending"` AND none of the
|
|
* above matched. The harness keeps structured diagnostic state on
|
|
* `toolState.agentDiagnostic`; `formatAgentHangBody` renders that as
|
|
* a markdown block.
|
|
*
|
|
* 6. Default — a generic `❌ Pullfrog failed` block with the raw error
|
|
* message in a fenced code block. Same body for both surfaces.
|
|
*
|
|
* The hang body and the API-key body diverge between the two surfaces only
|
|
* in that the job summary wraps them in the `### ❌ Pullfrog failed` H3
|
|
* banner; the PR comment uses the bare body since it already has Pullfrog
|
|
* branding in its footer.
|
|
*/
|
|
|
|
import type { AgentDiagnostic } from "./agentHangReport.ts";
|
|
import { formatAgentHangBody } from "./agentHangReport.ts";
|
|
import { formatApiKeyErrorSummary, isApiKeyAuthError } from "./apiKeys.ts";
|
|
import { BillingError, formatBillingErrorSummary } from "./billingErrors.ts";
|
|
import {
|
|
extractProviderId,
|
|
isProviderBillingExhausted,
|
|
isRouterKeylimitExhaustedError,
|
|
} from "./providerErrors.ts";
|
|
|
|
export type RenderedRunError = {
|
|
summary: string;
|
|
comment: string;
|
|
};
|
|
|
|
function isProviderModelNotFoundError(message: string): boolean {
|
|
return message.includes("ProviderModelNotFoundError");
|
|
}
|
|
|
|
/**
|
|
* Best-known billing top-up URL per provider. Conservative list: only
|
|
* providers we've actually classified billing-exhaustion shapes for in
|
|
* `providerErrors.ts`. Unknown providers fall through to a generic CTA.
|
|
*/
|
|
const PROVIDER_BILLING_URLS: Record<string, string> = {
|
|
deepseek: "https://platform.deepseek.com/top_up",
|
|
anthropic: "https://console.anthropic.com/settings/billing",
|
|
openai: "https://platform.openai.com/account/billing",
|
|
google: "https://aistudio.google.com/usage",
|
|
opencode: "https://opencode.ai/zen",
|
|
};
|
|
|
|
/**
|
|
* `extractProviderId` only fires when the harness emits `providerID=...`
|
|
* (OpenCode log shape). Direct-provider errors (e.g. Anthropic SDK throwing
|
|
* `"Your credit balance is too low to access the Anthropic API"`) carry no
|
|
* such tag, so map their distinctive copy to a provider id here so the
|
|
* dashboard link is reachable.
|
|
*
|
|
* Pattern is intentionally tight (Anthropic-specific phrasing only) to
|
|
* avoid mis-tagging non-Anthropic billing-exhausted errors that happen to
|
|
* mention `"Anthropic API"` in passing — the broader phrase appears in
|
|
* fallback-chain agent prompt text and OpenCode harness logs.
|
|
*/
|
|
function detectProviderId(message: string): string | null {
|
|
const harnessId = extractProviderId(message);
|
|
if (harnessId) return harnessId;
|
|
if (/credit balance is too low/i.test(message)) return "anthropic";
|
|
return null;
|
|
}
|
|
|
|
function formatProviderBillingExhausted(input: { errorMessage: string }): string {
|
|
const providerId = detectProviderId(input.errorMessage);
|
|
const dashboardUrl = providerId ? PROVIDER_BILLING_URLS[providerId] : undefined;
|
|
|
|
const headline = providerId
|
|
? `**Your \`${providerId}\` account is out of credit.**`
|
|
: "**Your provider account is out of credit.**";
|
|
const cta = dashboardUrl
|
|
? `[Top up \`${providerId}\` →](${dashboardUrl})`
|
|
: "Top up your provider account, then re-trigger Pullfrog.";
|
|
|
|
return [
|
|
headline,
|
|
"",
|
|
"Pullfrog detected a billing-exhausted response from your provider — the agent stopped before completing this run.",
|
|
"",
|
|
cta,
|
|
"",
|
|
`\`\`\`\n${input.errorMessage}\n\`\`\``,
|
|
].join("\n");
|
|
}
|
|
|
|
function formatProviderModelNotFoundSummary(input: {
|
|
owner: string;
|
|
name: string;
|
|
raw: string;
|
|
}): string {
|
|
return (
|
|
`Pullfrog's free fallback model is no longer available in OpenCode's catalog. ` +
|
|
`Add an API key for your configured model in the Pullfrog console for \`${input.owner}/${input.name}\`, ` +
|
|
`or contact support if this persists.\n\n` +
|
|
`\`\`\`\n${input.raw}\n\`\`\``
|
|
);
|
|
}
|
|
|
|
export function renderRunError(input: {
|
|
errorMessage: string;
|
|
repo: { owner: string; name: string };
|
|
agentDiagnostic: AgentDiagnostic | undefined;
|
|
}): RenderedRunError {
|
|
// reclassify mid-run OpenRouter "key budget exhausted" as BillingError so
|
|
// the user gets the same actionable copy as a /api/proxy-token 402.
|
|
const billingError = isRouterKeylimitExhaustedError(input.errorMessage)
|
|
? new BillingError(input.errorMessage, { code: "router_keylimit_exhausted" })
|
|
: null;
|
|
|
|
if (billingError) {
|
|
const body = formatBillingErrorSummary(billingError, input.repo.owner);
|
|
return { summary: body, comment: body };
|
|
}
|
|
|
|
// gated on isHang because the harness sets `agentDiagnostic` on entry, so
|
|
// any non-hang throw that hits the outer catch (e.g. post-success
|
|
// output_schema validator, or a late cleanup throw after the run already
|
|
// succeeded) would otherwise render "Pullfrog failed" with stale event
|
|
// counts and silently drop the real errorMessage.
|
|
const isHang =
|
|
input.errorMessage.startsWith("activity timeout") ||
|
|
input.errorMessage.startsWith("agent still pending");
|
|
const hangBody = isHang
|
|
? formatAgentHangBody({
|
|
diagnostic: input.agentDiagnostic,
|
|
isHang: true,
|
|
errorMessage: input.errorMessage,
|
|
})
|
|
: null;
|
|
|
|
// BYOK provider billing-exhausted (DeepSeek "Insufficient Balance",
|
|
// Anthropic "credit balance is too low", OpenCode Zen `CreditsError` /
|
|
// `FreeUsageLimitError`, Gemini "spending cap"). distinct from the Router
|
|
// billing branches above — Router uses `BillingError`, this uses the agent
|
|
// log payload classified by `isProviderBillingExhausted`. see #835.
|
|
//
|
|
// checked BEFORE api-key auth: providers commonly return 401 (DeepSeek,
|
|
// Gemini) or include `"API Error: 401"` in the error body for billing
|
|
// exhaustion, which `isApiKeyAuthError` would otherwise match — surfacing
|
|
// a "rotate your key" CTA when the actual fix is "top up credits".
|
|
if (isProviderBillingExhausted(input.errorMessage)) {
|
|
const body = formatProviderBillingExhausted({ errorMessage: input.errorMessage });
|
|
return { summary: `### ❌ Pullfrog failed\n\n${body}`, comment: body };
|
|
}
|
|
|
|
const apiKeySource = hangBody ?? input.errorMessage;
|
|
const apiKeyErrorSummary = isApiKeyAuthError(apiKeySource)
|
|
? formatApiKeyErrorSummary({
|
|
owner: input.repo.owner,
|
|
name: input.repo.name,
|
|
raw: apiKeySource,
|
|
})
|
|
: null;
|
|
|
|
if (apiKeyErrorSummary) {
|
|
return { summary: apiKeyErrorSummary, comment: apiKeyErrorSummary };
|
|
}
|
|
|
|
if (isProviderModelNotFoundError(input.errorMessage)) {
|
|
const body = formatProviderModelNotFoundSummary({
|
|
owner: input.repo.owner,
|
|
name: input.repo.name,
|
|
raw: input.errorMessage,
|
|
});
|
|
return { summary: body, comment: body };
|
|
}
|
|
|
|
if (hangBody) {
|
|
return {
|
|
summary: `### ❌ Pullfrog failed\n\n${hangBody}`,
|
|
comment: hangBody,
|
|
};
|
|
}
|
|
|
|
return {
|
|
summary: `### ❌ Pullfrog failed\n\n\`\`\`\n${input.errorMessage}\n\`\`\``,
|
|
comment: input.errorMessage,
|
|
};
|
|
}
|