Files
shockbot/utils/runLifecycle.ts
T
David Blass fe2746198c fix: 6 unaddressed log-audit / run-audit findings (#840)
* fix: 6 unaddressed log-audit / run-audit findings

- #836 + #818 (clerk middleware SyntaxError on action-runtime endpoints):
  narrow proxy.ts matcher to exclude /api/repo/<owner>/<repo>/run-context,
  /api/runtime/*, /api/proxy-token. these are server-to-server with their
  own auth and have no Clerk session to evaluate, so clerkMiddleware's
  decodeJwt throws turn into 500s on every request.

- #837 (npx EBADDEVENGINES on customer's package.json): change runCli's
  bootstrap cwd from $GITHUB_WORKSPACE to os.tmpdir() so npm v11+ doesn't
  enforce devEngines.packageManager from the customer's tree before our
  bootstrap can install pullfrog. CLI process.chdir's to payload.cwd
  internally, so the runtime work still happens in $GITHUB_WORKSPACE.

- #838 (createWorkflowDispatch silently dropped 39 user runs on a 5xx
  spike): add bounded retry (3 attempts, ~750ms total) on Octokit 5xx and
  network errors inside dispatchReservedRun. preserves the existing 422
  "Unexpected inputs" path. retry budget stays well under GitHub's 10s
  webhook redelivery window.

- #833 (bail() redirect("/signout") propagated NEXT_REDIRECT into webhook
  handlers, 40+ 500s/24h): drop the redirect side-effect; bail now just
  classifies bad-credentials as non-retryable and propagates. UI flows
  that wanted auto-signout on revoked tokens can detect it themselves;
  the side-effect was wrong for any non-page caller.

- #835 (BYOK provider billing-exhausted fell through to raw error
  renderer, 37 review-mode runs/24h with no PR-side signal):
  - extend providerErrors patterns with Anthropic "credit balance is too
    low" + extract isProviderBillingExhausted / extractProviderId helpers
  - add a renderer branch in runErrorRenderer.ts that names the provider
    (parsed from providerID=) and links to its billing dashboard
  - route handleAgentResult's !result.success path through the renderer
    + reportErrorToComment with createIfMissing, so review-mode and
    silent triggers get an actionable PR comment regardless of mode

- #834 (post-hook ERR_MODULE_NOT_FOUND already fixed on main, hardening):
  - add a vitest invariant that walks the entryPost.ts import graph and
    refuses any non-relative / non-node: specifier — catches the next
    `@actions/core` slip-up before publish
  - add an analyze-logs classifier so future entryPost crashes surface as
    failure:post-hook-module-not-found instead of hiding inside
    failure:unknown / failure:git-lock-file

Co-authored-by: Cursor <cursoragent@cursor.com>

* anneal: fix dead-code matcher, 404 url, silent-trigger gate, and grammar regression

Round-1 anneal pass on the audit-fixes PR surfaced two critical issues +
several majors that the original fixes shipped with:

- proxy.ts matcher 1 (`(?!_next|[^?]*\.(...))`) still caught every
  /api/... route because matcher arrays are OR'd. The narrowing in
  matcher 2 was dead code → middleware still 500'd on /api/runtime/*,
  /api/proxy-token, /api/repo/.../run-context. Carve-out now lives in
  BOTH matchers.

- opencode.ai/billing returns 404; canonical top-up surface is /zen.
  deepseek /usage is the consumption page, not the top-up flow (/top_up
  is correct). google /apikey is the keys list, not billing (/usage is
  the spend dashboard). All three URL strings updated.

- handleAgentResult gated reportErrorToComment behind `if (!ctx.silent)`,
  contradicting the createIfMissing intent — silent IncrementalReview /
  pull_request_synchronize / auto-label still got zero PR signal on BYOK
  billing exhaustion, the exact failure mode #835 was meant to fix.
  Moved createIfMissing into finalizeSuccessRun's existing render-and-
  post block (single source of truth), reverted handleAgentResult to its
  prior shape. Side benefit: drops the double-PATCH that fired on every
  non-silent !success path with an existing progress comment.

- Anthropic-direct error rendered "**Your your provider account is out
  of credit.**" because Anthropic SDK has no providerID= tag, so
  extractProviderId returned null and the headline composed
  "Your " + "your provider". Added detectProviderId Anthropic fallback
  (matches "Anthropic API" / "credit balance is too low") so the link
  is reachable AND made the headline conditional on whether a provider
  id was detected.

- Reordered renderRunError classifier: BYOK billing-exhausted now runs
  BEFORE api-key auth detection. Providers commonly return 401 for
  billing exhaustion (DeepSeek, Gemini), and the OpenCode harness logs
  often include "API Error: 401" in the raw error body, which
  isApiKeyAuthError would otherwise match — surfacing "rotate your key"
  when the actual fix is "top up credits".

- isTransientUpstreamError missed ENOTFOUND / ENETUNREACH / EHOSTUNREACH
  (undici DNS-class failures Octokit doesn't wrap with a status). Added
  to the prefix alternation.

- Tightened "10s webhook redelivery budget" / "GitHub redelivers"
  wording in triggerWorkflow.ts and bail.ts JSDoc — GitHub's 10s is the
  response timeout (it doesn't auto-redeliver); upstream webhook proxy
  retries are what multiplied the failure.

Co-authored-by: Cursor <cursoragent@cursor.com>

* anneal r2: unbreak proxy.ts matcher 2, extend carve-out, mkdtemp bootstrap

Round-2 anneal (security + cross-cutting lenses) on top of the round-1
audit-fixes commit caught a critical regression + several majors:

- proxy.ts matcher 2 from r1 (`/(api|trpc)(?!...)(.*)`) does NOT compile.
  path-to-regexp rejects a top-level `(?!` after `)` as "Pattern cannot
  start with '?'", and Next.js's SourceSchema runs the same validator at
  build time and aborts via `process.exit(1)`. PR #840's Vercel + preview
  deployments have been failing since 978eca26 for exactly this reason.
  Fixed by nesting the lookahead inside an outer parameter group:
  `/(api|trpc)((?!...).*)`. Same shape matcher 1 already uses, which is
  why m1 always compiled. Verified `pnpm next build` succeeds end-to-end.

- proxy.ts carve-out was incomplete relative to its own justification.
  The "no Clerk session, decodeJwt 500" failure mode applies to ALL
  server-to-server action-runtime endpoints — five more share the exact
  shape: /api/repo/.../learnings, /api/repo/.../pr/.../summary-comment,
  /api/repo/.../issue/.../plan-comment, /api/workflow-run/, and
  /api/github/installation-token. Extended both matchers. /api/upload/
  signed-url stays in (dual auth: Clerk session OR bearer JWT — needs
  middleware for the user path).

- proxy.ts carve-outs were unanchored: a future /api/proxy-token-info,
  /api/proxy-tokens, or /api/repo/X/Y/run-context-foo would silently
  bypass Clerk. Added `(?:$|/)` for path-prefix carve-outs (allow exact
  match or sub-path), `$` for routes with browser-callable siblings
  (e.g. `learnings/history` is browser-Clerk, `learnings$` is action-
  bearer-JWT). Verified 30/31 routes via path-to-regexp test harness.

- runCli.ts cwd flipped from $GITHUB_WORKSPACE to os.tmpdir() in r1
  (#837 fix for npm v11 devEngines.packageManager EBADDEVENGINES). But
  $TMPDIR is overridable from a prior $GITHUB_ENV step — a customer-
  authored or compromised prior step can plant /atk/node_modules/
  pullfrog/ and `echo "TMPDIR=/atk" >> $GITHUB_ENV`, and our npx
  --yes pullfrog@<v> bootstrap resolves the local install first,
  executing attacker code with full action env (provider keys, OIDC,
  installation token, CODEX_AUTH_JSON). Switched to mkdtempSync(join
  (tmpdir(), "pullfrog-bootstrap-")) — fresh per-invocation 0700 dir,
  not pre-writable by anything earlier in the job.

- runLifecycle.ts: writeRunErrorOutputs (catch-path) didn't pass
  createIfMissing: true, contradicting the symmetric intent of the
  r1 finalizeSuccessRun fix. Silent triggers (IncrementalReview /
  pull_request_synchronize / auto-label) that throw past the success
  path still got zero PR signal — exact failure mode #835 was meant
  to close. Now both paths pass createIfMissing: true.

- runErrorRenderer.ts detectProviderId regex `/Anthropic API|credit
  balance is too low/i` could mis-tag a non-Anthropic billing-exhausted
  error that mentioned "Anthropic API" in passing (fallback-chain agent
  prompt text, OpenCode harness logs). Tightened to /credit balance is
  too low/i — Anthropic-specific phrasing, sufficient for the direct-
  Anthropic SDK case the fallback exists to handle.

- runErrorRenderer.ts JSDoc: r1's classifier reorder put hang at #6 in
  code but the JSDoc still listed it at #2. Reordered the doc to match
  dispatch order, with explicit note that hang is a sub-source for the
  api-key check (which is why hangBody is precomputed early).

- analyze-logs.ts: ERR_MODULE_NOT_FOUND.*entryPost regex needs `s`
  flag so it survives Node v23+ stack-trace reformatting onto multiple
  lines. One-char fix to defend the #834 classification bucket.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 16:53:18 +00:00

183 lines
7.6 KiB
TypeScript

/**
* End-of-run cleanup phases extracted out of `main.ts`. Three shapes:
*
* - `persistRunArtifacts`: best-effort post-review cleanup + summary +
* learnings persistence. Shared by both the success path and the
* error-catch path; idempotent (each step has its own guard against
* double-execution).
*
* - `finalizeSuccessRun`: success-only — calls `persistRunArtifacts`
* first, then surfaces harness-side failures in the progress comment,
* deletes stranded progress comments, writes the GitHub Actions job
* summary, and emits the structured output marker.
*
* - `writeRunErrorOutputs`: error-only — writes the rendered error
* summary to the Actions summary tab and mirrors it to the PR
* progress comment. The catch path calls this and then
* `persistRunArtifacts` separately so the rendered error lands before
* the persistence calls, in case the latter throw.
*
* All three swallow their own non-fatal errors (`log.debug` or empty
* `catch {}`) so a cleanup failure can't flip an already-decided run
* outcome.
*/
import * as core from "@actions/core";
import type { AgentResult } from "../agents/shared.ts";
import { deleteProgressComment } from "../mcp/comment.ts";
import type { ToolContext } from "../mcp/server.ts";
import type { ToolState } from "../toolState.ts";
import { formatUsageSummary, log, writeSummary } from "./cli.ts";
import { reportErrorToComment } from "./errorReport.ts";
import { persistLearnings } from "./learnings.ts";
import { persistSummary } from "./prSummary.ts";
import { postReviewCleanup } from "./reviewCleanup.ts";
import { type RenderedRunError, renderRunError } from "./runErrorRenderer.ts";
/**
* Best-effort cleanup shared by both run-end paths:
* 1. post-review cleanup (dispatch follow-up re-review on submitted reviews)
* 2. persist the agent-edited PR summary tmpfile
* 3. persist the agent-edited repo-level learnings tmpfile
*
* Each step is idempotent and swallows its own errors. Safe to call from
* both `main()`'s success path and its catch path.
*/
export async function persistRunArtifacts(toolContext: ToolContext): Promise<void> {
await postReviewCleanup(toolContext).catch((error) => {
log.debug(`post-review cleanup failed: ${error}`);
});
await persistSummary(toolContext);
await persistLearnings(toolContext);
}
/**
* Run the success-path cleanup waterfall:
*
* 1. shared best-effort cleanup via `persistRunArtifacts`
* 2. when the harness returned `success=false` (e.g. unsubmitted-review
* gate exhausted retries, stop-hook persistently failing), render via
* `renderRunError` and surface the error in BOTH the progress comment
* (rendered.comment) and the Actions job summary (rendered.summary,
* prepended below in step 4) — same classifier as the catch path so
* the user sees it instead of a deleted-comment void / empty summary
* tab
* 3. when the run succeeded and the progress comment was never finalized
* via `report_progress`, delete it (three sub-cases — orphan
* "Leaping into action" comment, abandoned checklist, agent wrote
* a substantive artifact via another MCP write tool but skipped
* report_progress)
* 4. write the GitHub Actions step summary (best-effort — a write
* failure must not throw past this point because we'd hit the outer
* catch and clobber any progress comment we just wrote)
* 5. emit the structured output marker for tests + workflow consumers
*/
export async function finalizeSuccessRun(input: {
toolContext: ToolContext;
toolState: ToolState;
result: AgentResult;
repo: { owner: string; name: string };
}): Promise<void> {
await persistRunArtifacts(input.toolContext);
// shared rendering for the !success branch — same classifier as the
// outer catch path (BillingError reclassify → hang → BYOK billing →
// api-key → generic), so a harness-returned `{success: false}` lands an
// actionable error block in the job summary alongside the matching body
// in the progress comment. hang and generic get the `### ❌ Pullfrog
// failed` H3 banner; BillingError, BYOK billing, and api-key render
// their own provider-specific framing (no banner). renders once; reused
// for both surfaces below.
const rendered = !input.result.success
? renderRunError({
errorMessage: input.result.error || "agent run failed",
repo: input.repo,
agentDiagnostic: input.toolState.agentDiagnostic,
})
: null;
// `createIfMissing: true` is load-bearing for silent triggers
// (IncrementalReview / pull_request_synchronize / auto-label) that have
// no progress comment to update — without it, terminal failures like
// BYOK billing exhaustion land only in the GH job summary, which most
// users never open. `reportErrorToComment` no-ops when both progress
// comment AND issue context are absent. see #835.
if (rendered) {
await reportErrorToComment({
toolState: input.toolState,
error: rendered.comment,
createIfMissing: true,
}).catch((error) => {
log.debug(`failure error report failed: ${error}`);
});
}
// create_pull_request_review owns its own deletion (see mcp/review.ts), so
// progressComment is already null by the time we get here for that path.
// uses finalSummaryWritten (not todoTracker.enabled or wasUpdated) so
// cleanup survives API failures in report_progress where cancel() ran but
// the write didn't succeed, and isn't fooled by writes to *other* artifacts.
if (
input.result.success &&
input.toolState.progressComment &&
!input.toolState.finalSummaryWritten
) {
await deleteProgressComment(input.toolContext).catch((error) => {
log.debug(`stranded progress comment cleanup failed: ${error}`);
});
}
try {
const usageSummary = formatUsageSummary(input.toolState.usageEntries);
const body = input.toolState.lastProgressBody || input.result.output;
const parts = [rendered?.summary, body, usageSummary].filter(Boolean);
if (parts.length > 0) {
await writeSummary(parts.join("\n\n"));
}
} catch (error) {
log.debug(`job summary write failed: ${error}`);
}
if (input.toolState.output) {
log.info(`::pullfrog-output::${Buffer.from(input.toolState.output).toString("base64")}`);
core.setOutput("result", input.toolState.output);
}
}
/**
* Write the rendered error to the GitHub Actions job summary tab + mirror
* to the PR progress comment when one exists. Catch path only.
*
* `lastProgressBody` and the usage table are appended to the summary so the
* partial work the agent did before failing isn't lost.
*
* `createIfMissing: true` is symmetric with `finalizeSuccessRun` — silent
* triggers (IncrementalReview / pull_request_synchronize / auto-label) that
* throw past `finalizeSuccessRun` (e.g. timeout race kills the agent
* mid-billing-exhausted-retry) reach this catch path with no progress
* comment to update, and without `createIfMissing` the terminal error
* lands only in the GH job summary that most users never open. see #835.
*/
export async function writeRunErrorOutputs(input: {
rendered: RenderedRunError;
toolState: ToolState;
}): Promise<void> {
try {
const usageSummary = formatUsageSummary(input.toolState.usageEntries);
const parts = [input.rendered.summary, input.toolState.lastProgressBody, usageSummary].filter(
Boolean
);
await writeSummary(parts.join("\n\n"));
} catch {}
try {
await reportErrorToComment({
toolState: input.toolState,
error: input.rendered.comment,
createIfMissing: true,
});
} catch {
// error reporting failed, but don't let it mask the original error
}
}