f662b1a0c8
* unify per-run token + cost accounting across agents every agent harness now logs the same 5-column (or 6 with cost) table and populates the same AgentUsage contract, regardless of agent or upstream provider. previously OpenCode and the Claude fallback path emitted a 3-col table whose "Input Tokens" was actually only the non-cached delta, silently dropping cache read/write — real runs were being reported at ~0.4% of their true input (e.g. one baseline showed Input=30 while step_finish events summed to cache_read=724,753). changes: - add logTokenTable helper in action/agents/shared.ts with stable columns: Input | Cache Read | Cache Write | Output | Total | Cost ($). cost column renders only when a value is known. - action/agents/opencode.ts: accumulate step_finish.part.tokens AND step_finish.part.cost (sourced from models.dev inside opencode — confirmed working across Anthropic, OpenAI, Google, xAI, DeepSeek, Moonshot, and OpenRouter). drop the event.stats.total_tokens fallback since that payload has no cache breakdown. - action/agents/claude.ts: success-path now treats input_tokens as the non-cached field (matching OpenCode semantics), carries cache_read_input_tokens / cache_creation_input_tokens separately, and captures total_cost_usd from the final result event. the per-message fallback accumulator now captures cache fields too so it's no longer lossy when the result event never fires. - formatUsageSummary gains a Cost ($) column that matches the stdout table row-for-row; missing values render as "—". - scripts/token-usage.ts parses all three historical formats (new 5-col, legacy 4-col Claude success, legacy 3-col lossy) and explicitly flags the lossy runs instead of averaging misleading values. validation (pnpm play --local, identical "say hello" prompt): agent+model Input CacheR CacheW Output Total Cost OpenCode + Anthropic Sonnet 4.6 4 41,177 20,735 129 62,045 $0.0921 Claude CLI + Anthropic Sonnet 4.6 9 80,133 11,611 389 92,142 $0.0766 OpenCode + OpenAI codex-mini 10,893 46,976 0 606 58,475 $0.0059 OpenCode + Google Gemini 3 Flash — — — — — $0.0114 OpenCode + xAI Grok 4 Fast — — — — — $0.0035 OpenCode + DeepSeek Chat 18,854 0 0 1 18,855 $0.0053 OpenCode + Moonshot Kimi K2.5 — — — — — $0.0106 OpenCode + OpenRouter→Anthropic — — — — — $0.0617 OpenCode + OpenRouter→OpenAI — — — — — $0.0038 * isolate play.ts from developer gitconfig play.ts is a CI-emulator but inherits the developer's user- and system-scope gitconfig. a common local convenience — url."git@github.com:".insteadOf "https://github.com/" to force SSH auth — gets applied at read time on every git call inside the temp repo, causing `git remote get-url --push origin` to return an SSH URL instead of the stored HTTPS one. pullfrog_push_branch's validatePushDestination (correctly) treats that as tampering and blocks the push. the agent then burns the full MAX_COMMIT_RETRIES budget trying workarounds that can't beat a user-scope insteadOf rule, turning a trivial "say hello" run into a 1.35M-token session. point GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at /dev/null inside run() so the play process and its spawned agent see the same empty gitconfig that a real CI runner would. CI has no rewrites, so this is a no-op there; dev machines get CI-identical git state. SSH client config (~/.ssh/config and keys) is separate from gitconfig and is unaffected, so setupTestRepo's SSH clone still works locally. setupGit only writes --local scope, so nothing downstream depends on user-scope values. verification: with the scratch repo cleaned up and this isolation in place, OpenCode + Anthropic on the same "say hello" prompt goes from 1,349,654 tokens / $2.00+ to 62,045 tokens / $0.0921 — no retry loop, no push blocks. * persist aggregated token + cost usage to WorkflowRun AgentUsage has been memory-only — rendered into the GitHub step summary and then discarded when the runner tears down. that made questions like "avg cost per customer per day" require log-spelunking. persist it: - add Int? columns for inputTokens / outputTokens / cacheReadTokens / cacheWriteTokens and a Decimal? costUsd column on workflow_runs. Int4's 2.1B ceiling is ~200x larger than any realistic run so BigInt would be overkill. costUsd uses the same default Decimal precision as existing money columns (accounts.usageUsd, proxy_keys.hwmUsage). - extend PATCH /api/workflow-run/[runId] to accept the new numeric fields alongside the existing artifact strings. per-field type validation ensures the allowlist stays scalar-safe and rejects negative / non-finite values. - generalize patchWorkflowRunFields in the action so it accepts a mixed string/number payload, and add an aggregateUsage(entries) helper that sums per-agent AgentUsage records into a single patch. - call the reporter from main.ts's outer finally block, gated on toolContext. this is the shared cleanup path that every agent implementation flows through — claude.ts, opencode.ts, and any future harness all push their AgentUsage into toolState.usageEntries via the same line 468, so one finally-block call covers them all. running in finally also means partial usage gets persisted even when the agent errored out mid-run. * anneal token + cost accounting follow-up polish from a review pass: - aggregate usage across commit-retry iterations inside each agent harness. previously runClaude / runOpenCode returned only the final retry's usage, so any run that hit the dirty-tree retry loop under-counted tokens and cost in both the stdout table and the WorkflowRun row. added a shared mergeAgentUsage helper in agents/shared.ts; both harnesses now fold each iteration's usage into a running total and return the sum. - scripts/token-usage.ts now handles the unified format with or without the Cost ($) column. previously the int-only number regex rejected decimals and the 5-cell length check rejected 6-cell rows, so logs from post-cost-tracking runs fell through to "no token table". the parser now accepts both 5- and 6-cell unified rows, splits int vs decimal cells, and averages reported Cost alongside the tokens. - PATCH /api/workflow-run/[runId] now rejects INT field values above INT4_MAX (2_147_483_647) so a malformed payload gets a clean 400 instead of propagating a Prisma error. also defends against a compromised runner sending a deliberately huge value. - clarifying comments: opencode.ts documents that step_finish.part.cost is a per-step delta (empirically verified), main.ts explains that toolState.usageEntries already carries merged per-retry usage so aggregateUsage just sums entries (one per agent.run()). - tests for aggregateUsage and mergeAgentUsage — 12 new cases covering empty / partial / multi-agent inputs and the "keep undefined" semantic that prevents spurious zeros from being persisted. - drop `as number` cast in logTokenTable — narrow via const instead. * anneal: clamp INT overflow + guarantee mergeAgentUsage immutability second review pass surfaced two defensive gaps: - a single token field exceeding INT4_MAX would pass the client but be rejected by the server's per-field validator, writing a partial row with some NULLs where sums belonged. clamp in aggregateUsage so the wire payload is always self-consistent across all numeric columns, with a loud warning so the clamp doesn't silently swallow weirdness. - mergeAgentUsage's single-sided branches returned the input reference. callers treat AgentUsage as immutable but future callers might not; always return a fresh shallow copy instead. two new tests guarantee the no-mutation-leak property. no behavior change in the happy path — INT4_MAX is ~200x the largest realistic per-run token count. * anneal: resilient usage persistence + cross-platform null device third review pass surfaced three small issues: - main.ts finally block: writeGitHubUsageSummaryToFile throwing would skip the WorkflowRun usage PATCH. both are independent best-effort cleanup tasks — wrap the former in catch so a filesystem failure doesn't block DB persistence. - AgentUsage.inputTokens had no jsdoc explaining that it's the full billable input (cached + non-cached). the same word "Input" means "non-cached only" in the stdout/markdown tables (derived by subtraction). document the semantic so dashboards querying WorkflowRun.inputTokens don't misinterpret it. - play.ts gitconfig isolation was hard-coded to "/dev/null" which doesn't exist on Windows. use `os.devNull` for cross-platform parity (resolves to `\\.\nul` on win32). the project is Linux-only in CI so this only helps local Windows contributors, but it's a zero-cost swap. also updated the finally-block caveat comment: usage is only pushed to toolState.usageEntries when agent.run() returns an AgentResult, not when the timeout race rejects — so timed-out runs don't persist partial usage. documented instead of trying to thread state through Promise.race. * anneal: NaN-guard cost accumulators + clarify inputTokens docs final polish from review round 4: - guard both cost accumulators (opencode step_finish.part.cost and claude result.total_cost_usd) with Number.isFinite. `typeof x === "number"` accepts NaN, and one NaN `+=` would poison the running total for the whole session. - reword prisma schema comment on WorkflowRun usage fields to call out that cacheReadTokens / cacheWriteTokens are SUB-totals within inputTokens (not additional tokens on top). prevents future dashboards from double-counting by ~2x when summing "total tokens used".
647 lines
25 KiB
TypeScript
647 lines
25 KiB
TypeScript
// changes to tool permissions should be reflected in wiki/granular-tools.md
|
|
|
|
import { existsSync, readdirSync } from "node:fs";
|
|
import { join } from "node:path";
|
|
import * as core from "@actions/core";
|
|
import { deleteProgressComment, reportProgress } from "./mcp/comment.ts";
|
|
import { startInstallation } from "./mcp/dependencies.ts";
|
|
import {
|
|
initToolState,
|
|
startMcpHttpServer,
|
|
type ToolContext,
|
|
type ToolState,
|
|
} from "./mcp/server.ts";
|
|
import { computeModes } from "./modes.ts";
|
|
import {
|
|
type ActivityTimeout,
|
|
createProcessOutputActivityTimeout,
|
|
DEFAULT_ACTIVITY_CHECK_INTERVAL_MS,
|
|
DEFAULT_ACTIVITY_TIMEOUT_MS,
|
|
} from "./utils/activity.ts";
|
|
import { resolveAgent, resolveModel } from "./utils/agent.ts";
|
|
import { apiFetch } from "./utils/apiFetch.ts";
|
|
import { validateAgentApiKey } from "./utils/apiKeys.ts";
|
|
import { resolveBody } from "./utils/body.ts";
|
|
import { formatUsageSummary, log, writeSummary } from "./utils/cli.ts";
|
|
import { recordDiffReadFromToolUse } from "./utils/diffCoverage.ts";
|
|
import { reportErrorToComment } from "./utils/errorReport.ts";
|
|
import { onExitSignal } from "./utils/exitHandler.ts";
|
|
import { resolveGit, setGitAuthServer } from "./utils/gitAuth.ts";
|
|
import { startGitAuthServer } from "./utils/gitAuthServer.ts";
|
|
import { createOctokit, writeGitHubUsageSummaryToFile } from "./utils/github.ts";
|
|
import { resolveInstructions } from "./utils/instructions.ts";
|
|
import { executeLifecycleHook } from "./utils/lifecycle.ts";
|
|
import { normalizeEnv } from "./utils/normalizeEnv.ts";
|
|
import { aggregateUsage, patchWorkflowRunFields } from "./utils/patchWorkflowRunFields.ts";
|
|
import { resolvePayload, resolvePromptInput } from "./utils/payload.ts";
|
|
import { postReviewCleanup } from "./utils/reviewCleanup.ts";
|
|
import { handleAgentResult } from "./utils/run.ts";
|
|
import { resolveRunContextData } from "./utils/runContextData.ts";
|
|
import { setEnvAllowlist } from "./utils/secrets.ts";
|
|
import { createTempDirectory, setupGit } from "./utils/setup.ts";
|
|
import { killTrackedChildren } from "./utils/subprocess.ts";
|
|
import { resolveTimeoutMs, TIMEOUT_DISABLED } from "./utils/time.ts";
|
|
import { Timer } from "./utils/timer.ts";
|
|
import { createTodoTracker } from "./utils/todoTracking.ts";
|
|
import { getJobToken, resolveTokens } from "./utils/token.ts";
|
|
import { resolveRun } from "./utils/workflow.ts";
|
|
|
|
export { Inputs } from "./utils/payload.ts";
|
|
|
|
export interface MainResult {
|
|
success: boolean;
|
|
output?: string | undefined;
|
|
error?: string | undefined;
|
|
result?: string | undefined;
|
|
}
|
|
|
|
function resolveOutputSchema(): Record<string, unknown> | undefined {
|
|
const raw = core.getInput("output_schema");
|
|
if (!raw) return undefined;
|
|
|
|
let parsed: unknown;
|
|
try {
|
|
parsed = JSON.parse(raw);
|
|
} catch {
|
|
throw new Error(`invalid output_schema: not valid JSON`);
|
|
}
|
|
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
|
throw new Error(`invalid output_schema: must be a JSON object`);
|
|
}
|
|
log.info("» structured output schema provided — output will be required");
|
|
return parsed as Record<string, unknown>;
|
|
}
|
|
|
|
function resolveTimeoutForLog(timeout: string | undefined): string {
|
|
if (!timeout) return "1h (default)";
|
|
if (timeout === TIMEOUT_DISABLED) return "none (disabled)";
|
|
return timeout;
|
|
}
|
|
|
|
function resolveModelForLog(ctx: {
|
|
payload: ResolvedPayload;
|
|
resolvedModel: string | undefined;
|
|
}): string {
|
|
const envModel = process.env.PULLFROG_MODEL?.trim();
|
|
if (envModel) return `${envModel} (override via PULLFROG_MODEL)`;
|
|
if (ctx.payload.proxyModel) return `${ctx.payload.proxyModel} (proxy)`;
|
|
if (ctx.resolvedModel && ctx.payload.model && ctx.payload.model !== ctx.resolvedModel) {
|
|
return `${ctx.resolvedModel} (resolved from ${ctx.payload.model})`;
|
|
}
|
|
if (ctx.resolvedModel) return ctx.resolvedModel;
|
|
if (ctx.payload.model) return `${ctx.payload.model} (unresolved)`;
|
|
return "auto";
|
|
}
|
|
|
|
function resolveAgentForLog(ctx: { agentName: string; resolvedModel: string | undefined }): string {
|
|
const envAgent = process.env.PULLFROG_AGENT?.trim();
|
|
if (envAgent && envAgent === ctx.agentName) {
|
|
return `${ctx.agentName} (override via PULLFROG_AGENT)`;
|
|
}
|
|
if (ctx.agentName === "claude" && ctx.resolvedModel) {
|
|
return `${ctx.agentName} (auto-selected for ${ctx.resolvedModel})`;
|
|
}
|
|
return ctx.agentName;
|
|
}
|
|
|
|
import type { ResolvedPayload } from "./utils/payload.ts";
|
|
|
|
interface OidcCredentials {
|
|
requestUrl: string;
|
|
requestToken: string;
|
|
}
|
|
|
|
async function mintProxyKey(ctx: { oidcCredentials: OidcCredentials }): Promise<string | null> {
|
|
try {
|
|
process.env.ACTIONS_ID_TOKEN_REQUEST_URL = ctx.oidcCredentials.requestUrl;
|
|
process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN = ctx.oidcCredentials.requestToken;
|
|
const oidcToken = await core.getIDToken("pullfrog-api");
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_URL;
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN;
|
|
|
|
const response = await apiFetch({
|
|
path: "/api/proxy-token",
|
|
method: "POST",
|
|
headers: { Authorization: `Bearer ${oidcToken}` },
|
|
});
|
|
|
|
if (!response.ok) {
|
|
log.warning(`proxy key mint failed (${response.status})`);
|
|
return null;
|
|
}
|
|
|
|
const data = (await response.json()) as { key: string };
|
|
return data.key;
|
|
} catch (error) {
|
|
log.warning(`proxy key mint error: ${error instanceof Error ? error.message : String(error)}`);
|
|
return null;
|
|
} finally {
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_URL;
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN;
|
|
}
|
|
}
|
|
|
|
async function resolveProxyModel(ctx: {
|
|
payload: ResolvedPayload;
|
|
oss: boolean;
|
|
proxyModel?: string | undefined;
|
|
oidcCredentials: OidcCredentials | null;
|
|
}): Promise<void> {
|
|
// env override = BYOK escape hatch, don't proxy
|
|
if (process.env.PULLFROG_MODEL?.trim()) return;
|
|
|
|
// OSS: server decided the model
|
|
if (ctx.oss && ctx.proxyModel) {
|
|
if (!ctx.oidcCredentials) {
|
|
log.warning("» oss repo but no OIDC credentials available — skipping proxy");
|
|
return;
|
|
}
|
|
const key = await mintProxyKey({ oidcCredentials: ctx.oidcCredentials });
|
|
if (!key) return;
|
|
|
|
process.env.OPENROUTER_API_KEY = key;
|
|
core.setSecret(key);
|
|
ctx.payload.proxyModel = ctx.proxyModel;
|
|
log.info(`» proxy: oss → ${ctx.proxyModel}`);
|
|
return;
|
|
}
|
|
|
|
// managed billing will add its path here later
|
|
}
|
|
|
|
async function writeJobSummary(toolState: ToolState): Promise<void> {
|
|
const usageSummary = formatUsageSummary(toolState.usageEntries);
|
|
const summaryParts = [toolState.lastProgressBody, usageSummary].filter(Boolean);
|
|
if (summaryParts.length > 0) {
|
|
await writeSummary(summaryParts.join("\n\n"));
|
|
}
|
|
}
|
|
|
|
export async function main(): Promise<MainResult> {
|
|
// normalize env var names to uppercase (handles case-insensitive workflow files)
|
|
normalizeEnv();
|
|
|
|
// write usage summary on SIGINT/SIGTERM so the worker can read it after sandbox.exec
|
|
const usageSummaryPath = process.env.PULLFROG_USAGE_SUMMARY_PATH;
|
|
if (usageSummaryPath) {
|
|
onExitSignal(() => writeGitHubUsageSummaryToFile(usageSummaryPath));
|
|
}
|
|
|
|
const timer = new Timer();
|
|
let activityTimeout: ActivityTimeout | null = null;
|
|
let safetyNetTimer: NodeJS.Timeout | undefined;
|
|
|
|
// parse prompt early to extract progressCommentId for toolState
|
|
const resolvedPromptInput = resolvePromptInput();
|
|
|
|
const toolState = initToolState({
|
|
progressCommentId:
|
|
typeof resolvedPromptInput !== "string" ? resolvedPromptInput.progressCommentId : undefined,
|
|
});
|
|
|
|
// resolve and fingerprint git binary before any agent code runs
|
|
resolveGit();
|
|
|
|
// get job token for initial API calls
|
|
const jobToken = getJobToken();
|
|
const initialOctokit = createOctokit(jobToken);
|
|
const runContext = await resolveRunContextData({ octokit: initialOctokit, token: jobToken });
|
|
timer.checkpoint("runContextData");
|
|
|
|
// inject account-level secrets into process.env (YAML secrets take precedence)
|
|
if (runContext.dbSecrets) {
|
|
for (const [key, value] of Object.entries(runContext.dbSecrets)) {
|
|
if (!process.env[key]) {
|
|
process.env[key] = value;
|
|
core.setSecret(value);
|
|
}
|
|
}
|
|
const count = Object.keys(runContext.dbSecrets).length;
|
|
if (count > 0) log.info(`» ${count} db secret(s) loaded`);
|
|
}
|
|
|
|
// configure env allowlist for subprocess filtering
|
|
if (runContext.repoSettings.envAllowlist) {
|
|
setEnvAllowlist(runContext.repoSettings.envAllowlist);
|
|
}
|
|
|
|
// resolve payload to determine shell permission
|
|
const payload = resolvePayload(resolvedPromptInput, runContext.repoSettings);
|
|
toolState.model = payload.model;
|
|
if (payload.event.trigger === "pull_request_synchronize") {
|
|
toolState.beforeSha = payload.event.before_sha;
|
|
}
|
|
|
|
// resolve tokens first — acquireNewToken needs OIDC env vars for token exchange
|
|
await using tokenRef = await resolveTokens({ push: payload.push });
|
|
|
|
// stash OIDC credentials in memory before wiping from process.env
|
|
// the agent's shell commands can't access JS variables, so this is safe
|
|
const oidcCredentials: OidcCredentials | null =
|
|
process.env.ACTIONS_ID_TOKEN_REQUEST_URL && process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN
|
|
? {
|
|
requestUrl: process.env.ACTIONS_ID_TOKEN_REQUEST_URL,
|
|
requestToken: process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN,
|
|
}
|
|
: null;
|
|
|
|
// clear OIDC env vars in restricted mode to prevent agent from minting tokens
|
|
if (payload.shell !== "enabled") {
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_URL;
|
|
delete process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN;
|
|
}
|
|
|
|
// proxy decision: mint an OpenRouter key for OSS repos (or later, managed billing)
|
|
await resolveProxyModel({
|
|
payload,
|
|
oss: runContext.oss,
|
|
proxyModel: runContext.proxyModel,
|
|
oidcCredentials,
|
|
});
|
|
|
|
// create octokit with MCP token for GitHub API calls
|
|
const octokit = createOctokit(tokenRef.mcpToken);
|
|
|
|
const runInfo = await resolveRun({ octokit });
|
|
let toolContext: ToolContext | undefined;
|
|
let progressCallbackDisabled = false;
|
|
let todoTracker: ReturnType<typeof createTodoTracker> | undefined;
|
|
|
|
try {
|
|
if (payload.cwd && process.cwd() !== payload.cwd) {
|
|
process.chdir(payload.cwd);
|
|
}
|
|
|
|
// resolve body - fetches body_html and converts to markdown if images present
|
|
// this ensures agents receive markdown with working signed image URLs
|
|
const originalBody = payload.event.body;
|
|
const resolvedBody = await resolveBody({
|
|
event: payload.event,
|
|
octokit,
|
|
repo: runContext.repo,
|
|
});
|
|
if (resolvedBody !== originalBody) {
|
|
payload.event.body = resolvedBody;
|
|
// also update prompt if original body was included there
|
|
if (originalBody && payload.prompt.includes(originalBody)) {
|
|
payload.prompt = payload.prompt.replace(originalBody, resolvedBody ?? "");
|
|
}
|
|
}
|
|
|
|
const tmpdir = createTempDirectory();
|
|
|
|
await using gitAuthServer = await startGitAuthServer(tmpdir);
|
|
setGitAuthServer(gitAuthServer);
|
|
|
|
const resolvedModel = payload.proxyModel ? undefined : resolveModel({ slug: payload.model });
|
|
const agent = resolveAgent({ model: resolvedModel });
|
|
|
|
validateAgentApiKey({
|
|
agent,
|
|
model: payload.proxyModel ?? resolvedModel ?? payload.model,
|
|
owner: runContext.repo.owner,
|
|
name: runContext.repo.name,
|
|
});
|
|
|
|
await setupGit({
|
|
gitToken: tokenRef.gitToken,
|
|
owner: runContext.repo.owner,
|
|
name: runContext.repo.name,
|
|
octokit,
|
|
toolState,
|
|
shell: payload.shell,
|
|
postCheckoutScript: runContext.repoSettings.postCheckoutScript,
|
|
});
|
|
timer.checkpoint("git");
|
|
|
|
// execute setup lifecycle hook (runs once at initialization).
|
|
// setup is load-bearing — if it fails the rest of the run is in an
|
|
// undefined state, so upgrade the soft-fail warning to a hard error.
|
|
const setupHook = await executeLifecycleHook({
|
|
event: "setup",
|
|
script: runContext.repoSettings.setupScript,
|
|
});
|
|
if (setupHook.warning) {
|
|
throw new Error(setupHook.warning);
|
|
}
|
|
timer.checkpoint("lifecycleHooks::setup");
|
|
|
|
const agentId = agent.name;
|
|
const modes = [...computeModes(agentId), ...runContext.repoSettings.modes];
|
|
|
|
const outputSchema = resolveOutputSchema();
|
|
|
|
// mcpServerUrl and tmpdir are set after server starts
|
|
toolContext = {
|
|
agentId,
|
|
repo: runContext.repo,
|
|
payload,
|
|
octokit,
|
|
githubInstallationToken: tokenRef.mcpToken,
|
|
gitToken: tokenRef.gitToken,
|
|
apiToken: runContext.apiToken,
|
|
modes,
|
|
postCheckoutScript: runContext.repoSettings.postCheckoutScript,
|
|
prepushScript: runContext.repoSettings.prepushScript,
|
|
prApproveEnabled: runContext.repoSettings.prApproveEnabled,
|
|
modeInstructions: runContext.repoSettings.modeInstructions,
|
|
toolState,
|
|
runId: runInfo.runId,
|
|
jobId: runInfo.jobId,
|
|
mcpServerUrl: "",
|
|
tmpdir,
|
|
resolvedModel,
|
|
};
|
|
await using mcpHttpServer = await startMcpHttpServer(toolContext, { outputSchema });
|
|
toolContext.mcpServerUrl = mcpHttpServer.url;
|
|
log.info(`» MCP server started at ${mcpHttpServer.url}`);
|
|
timer.checkpoint("mcpServer");
|
|
|
|
startInstallation(toolContext);
|
|
|
|
const modelForLog = resolveModelForLog({ payload, resolvedModel });
|
|
const agentForLog = resolveAgentForLog({ agentName: agent.name, resolvedModel });
|
|
const timeoutForLog = resolveTimeoutForLog(payload.timeout);
|
|
log.info(`» model: ${modelForLog}`);
|
|
log.info(`» agent: ${agentForLog}`);
|
|
log.info(`» push: ${payload.push}`);
|
|
log.info(`» shell: ${payload.shell}`);
|
|
log.info(`» timeout: ${timeoutForLog}`);
|
|
|
|
const instructions = resolveInstructions({
|
|
payload,
|
|
repo: runContext.repo,
|
|
modes,
|
|
agentId,
|
|
outputSchema,
|
|
learnings: runContext.repoSettings.learnings,
|
|
});
|
|
const logParts = [
|
|
instructions.eventInstructions
|
|
? `EVENT-LEVEL INSTRUCTIONS:\n${instructions.eventInstructions}`
|
|
: null,
|
|
instructions.user ? `USER REQUEST:\n${instructions.user}` : null,
|
|
instructions.event,
|
|
].filter(Boolean);
|
|
log.box(logParts.join("\n\n---\n\n"), {
|
|
title: "Instructions",
|
|
});
|
|
log.group("View full prompt", () => {
|
|
log.info(instructions.full);
|
|
});
|
|
|
|
// OpenCode loads .opencode/plugin/ files at startup. if the repo has any,
|
|
// eagerly await dependency installation so plugin imports can resolve.
|
|
if (agentId === "opencode") {
|
|
const pluginDir = join(process.cwd(), ".opencode", "plugin");
|
|
const hasPlugins =
|
|
existsSync(pluginDir) && readdirSync(pluginDir).some((f) => /\.[jt]sx?$/.test(f));
|
|
if (hasPlugins && toolState.dependencyInstallation?.promise) {
|
|
log.info(
|
|
"» .opencode/plugin/ detected — awaiting dependency installation before agent start"
|
|
);
|
|
await toolState.dependencyInstallation.promise.catch(() => {});
|
|
timer.checkpoint("awaitDepsForPlugins");
|
|
}
|
|
}
|
|
|
|
// run agent, optionally with timeout enforcement
|
|
activityTimeout = createProcessOutputActivityTimeout({
|
|
timeoutMs: DEFAULT_ACTIVITY_TIMEOUT_MS,
|
|
checkIntervalMs: DEFAULT_ACTIVITY_CHECK_INTERVAL_MS,
|
|
});
|
|
activityTimeout.promise.catch(() => {}); // prevent unhandled rejection if agent wins race
|
|
todoTracker = createTodoTracker(async (body) => {
|
|
if (progressCallbackDisabled || !toolContext) return;
|
|
try {
|
|
await reportProgress(toolContext, { body });
|
|
} catch (err) {
|
|
log.debug(`progress update failed: ${err}`);
|
|
}
|
|
});
|
|
toolState.todoTracker = todoTracker;
|
|
|
|
// when the agent subprocess is killed for inner activity timeout, stop
|
|
// the MCP HTTP server so mcp-proxy's SSE reconnect attempts don't keep
|
|
// the outer activity timer alive. start a short safety-net timer — if
|
|
// the agent promise hasn't resolved within 5min after the inner kill,
|
|
// force-reject the outer timer so the run can exit.
|
|
let innerTimeoutFired = false;
|
|
const onInnerActivityTimeout = () => {
|
|
if (innerTimeoutFired) return;
|
|
innerTimeoutFired = true;
|
|
log.info(
|
|
"» inner activity timeout fired — stopping MCP server and starting 5min safety-net timer"
|
|
);
|
|
// fire and forget — the server's dispose is idempotent so the
|
|
// `await using` cleanup at block exit is still safe.
|
|
mcpHttpServer[Symbol.asyncDispose]().catch((err) => {
|
|
log.debug(
|
|
`mcp server stop after inner kill failed: ${err instanceof Error ? err.message : String(err)}`
|
|
);
|
|
});
|
|
safetyNetTimer = setTimeout(
|
|
() => {
|
|
activityTimeout?.forceReject(
|
|
"agent still pending 5min after inner activity kill — forcing exit"
|
|
);
|
|
},
|
|
5 * 60 * 1000
|
|
);
|
|
safetyNetTimer.unref?.();
|
|
};
|
|
|
|
const agentPromise = agent.run({
|
|
payload,
|
|
resolvedModel,
|
|
mcpServerUrl: mcpHttpServer.url,
|
|
tmpdir,
|
|
instructions,
|
|
todoTracker,
|
|
onActivityTimeout: onInnerActivityTimeout,
|
|
onToolUse: (event) => {
|
|
const wasTracked = recordDiffReadFromToolUse({
|
|
state: toolState.diffCoverage,
|
|
toolName: event.toolName,
|
|
input: event.input,
|
|
cwd: process.cwd(),
|
|
});
|
|
if (!wasTracked) return;
|
|
const trackedRanges = toolState.diffCoverage?.coveredRanges ?? [];
|
|
log.debug(
|
|
`» diff coverage tracked from tool ${event.toolName} (${trackedRanges.length} merged range${trackedRanges.length === 1 ? "" : "s"})`
|
|
);
|
|
},
|
|
});
|
|
// symmetric with the activityTimeout/timeoutPromise catches below: if a
|
|
// timeout wins the race, agentPromise is stranded and its later rejection
|
|
// becomes an unhandled rejection. node 15+ terminates the process on
|
|
// unhandled rejection by default, which would kill main() mid-cleanup and
|
|
// lose the error-reporting / usage-summary work that follows. the race
|
|
// still sees the rejection (the original promise is shared); this catch
|
|
// only keeps node from treating a post-race rejection as unobserved.
|
|
agentPromise.catch(() => {});
|
|
|
|
// timeout enforcement: default is 1 hour, but can be overridden via flags in the prompt:
|
|
// - --timeout=2h (or any duration like "--timeout=30m", "--timeout=1h30m") to set a custom timeout
|
|
// - --notimeout to disable timeout entirely
|
|
let result: Awaited<typeof agentPromise>;
|
|
if (payload.timeout === TIMEOUT_DISABLED) {
|
|
result = await Promise.race([agentPromise, activityTimeout.promise]);
|
|
} else {
|
|
// resolveTimeoutMs rejects unparseable / zero / setTimeout-overflow inputs
|
|
// so a bad string can't silently resolve to an instant timeout. fall back
|
|
// to the 1h default with a warning — users who want runtime measured in
|
|
// weeks should use --notimeout.
|
|
const usable = resolveTimeoutMs(payload.timeout);
|
|
if (payload.timeout && usable === null) {
|
|
log.warning(`invalid timeout "${payload.timeout}" (use --notimeout to disable), using 1h`);
|
|
}
|
|
const timeoutMs = usable ?? 3600000;
|
|
const actualTimeout = usable !== null ? payload.timeout : "1h";
|
|
let timeoutId: NodeJS.Timeout | undefined;
|
|
const timeoutPromise = new Promise<never>((_, reject) => {
|
|
timeoutId = setTimeout(() => {
|
|
reject(new Error(`agent run timed out after ${actualTimeout}`));
|
|
}, timeoutMs);
|
|
});
|
|
timeoutPromise.catch(() => {}); // prevent unhandled rejection if agent wins race
|
|
try {
|
|
result = await Promise.race([agentPromise, timeoutPromise, activityTimeout.promise]);
|
|
} finally {
|
|
clearTimeout(timeoutId);
|
|
}
|
|
}
|
|
|
|
// accumulate top-level agent usage
|
|
if (result.usage) {
|
|
toolState.usageEntries.push(result.usage);
|
|
}
|
|
|
|
// validate this before writing job summary to avoid masking the error
|
|
if (outputSchema && !toolState.output) {
|
|
throw new Error(
|
|
"output_schema was provided but agent did not call set_output — structured output is required"
|
|
);
|
|
}
|
|
|
|
// post-agent review cleanup: reportReviewNodeId → follow-up re-review dispatch.
|
|
// runs after the agent exits so ordering is architecturally guaranteed (no LLM involvement).
|
|
// best-effort: cleanup failures must not turn a successful agent run into a failure.
|
|
if (toolContext) {
|
|
await postReviewCleanup(toolContext).catch((error) => {
|
|
log.debug(`post-review cleanup failed: ${error}`);
|
|
});
|
|
}
|
|
|
|
// review submitted → always delete the progress comment.
|
|
// the review is the durable artifact; the progress comment is noise.
|
|
// defense-in-depth: covers the case where the agent calls report_progress
|
|
// despite mode instructions, which sets finalSummaryWritten and prevents
|
|
// the stranded-comment heuristic below from firing.
|
|
if (toolContext && toolState.review && toolState.progressCommentId) {
|
|
await deleteProgressComment(toolContext).catch((error) => {
|
|
log.debug(`review progress comment cleanup failed: ${error}`);
|
|
});
|
|
}
|
|
|
|
// clean up stranded progress comments. two cases:
|
|
// 1. wasUpdated=false: nothing wrote to the comment ("Leaping into action" orphan)
|
|
// 2. tracker published a checklist but the agent never wrote a final summary
|
|
// (hasPublished=true, finalSummaryWritten=false).
|
|
// in both cases, delete the comment so it doesn't linger with stale content.
|
|
// wasUpdated is intentionally NOT set here — cleanup is not a real progress update.
|
|
// uses finalSummaryWritten (not todoTracker.enabled) so cleanup survives API failures
|
|
// in report_progress where cancel() ran but the write didn't succeed.
|
|
const trackerWasLastWriter = todoTracker?.hasPublished && !toolState.finalSummaryWritten;
|
|
if (
|
|
toolContext &&
|
|
toolState.progressCommentId &&
|
|
(!toolState.wasUpdated || trackerWasLastWriter)
|
|
) {
|
|
await deleteProgressComment(toolContext).catch((error) => {
|
|
log.debug(`stranded progress comment cleanup failed: ${error}`);
|
|
});
|
|
}
|
|
|
|
await writeJobSummary(toolState);
|
|
|
|
// emit structured output marker for test validation
|
|
if (toolState.output) {
|
|
log.info(`::pullfrog-output::${Buffer.from(toolState.output).toString("base64")}`);
|
|
core.setOutput("result", toolState.output);
|
|
}
|
|
|
|
return await handleAgentResult({
|
|
result,
|
|
toolState,
|
|
silent: payload.event.silent ?? false,
|
|
});
|
|
} catch (error) {
|
|
const errorMessage = error instanceof Error ? error.message : "unknown error occurred";
|
|
progressCallbackDisabled = true;
|
|
todoTracker?.cancel();
|
|
killTrackedChildren();
|
|
log.error(errorMessage);
|
|
|
|
// best-effort summary — write the error so it's visible in the Actions summary tab
|
|
try {
|
|
const errorSummary = `### ❌ Pullfrog failed\n\n\`\`\`\n${errorMessage}\n\`\`\``;
|
|
const usageSummary = formatUsageSummary(toolState.usageEntries);
|
|
const parts = [errorSummary, toolState.lastProgressBody, usageSummary].filter(Boolean);
|
|
await writeSummary(parts.join("\n\n"));
|
|
} catch {}
|
|
|
|
try {
|
|
await reportErrorToComment({ toolState, error: errorMessage });
|
|
} catch {
|
|
// error reporting failed, but don't let it mask the original error
|
|
}
|
|
|
|
// best-effort review cleanup (e.g., agent timed out after submitting a review)
|
|
if (toolContext) {
|
|
await postReviewCleanup(toolContext).catch((error) => {
|
|
log.debug(`post-review cleanup failed: ${error}`);
|
|
});
|
|
}
|
|
|
|
return {
|
|
success: false,
|
|
error: errorMessage,
|
|
};
|
|
} finally {
|
|
activityTimeout?.stop();
|
|
if (safetyNetTimer) clearTimeout(safetyNetTimer);
|
|
if (usageSummaryPath) {
|
|
// a write error here (ENOSPC, EACCES, dirname removed) must not mask
|
|
// either the try's successful return or the catch's error return.
|
|
// the summary is informational — log and move on.
|
|
try {
|
|
await writeGitHubUsageSummaryToFile(usageSummaryPath);
|
|
} catch (err) {
|
|
log.debug(
|
|
`failed to write usage summary to ${usageSummaryPath}: ${err instanceof Error ? err.message : String(err)}`
|
|
);
|
|
}
|
|
}
|
|
|
|
// persist aggregated token + cost usage to the WorkflowRun row.
|
|
// this is the single shared cleanup path across every agent implementation:
|
|
// each agent harness returns a single AgentUsage from agent.run() that
|
|
// already aggregates its internal retries via mergeAgentUsage, and the
|
|
// success branch above pushes that entry into toolState.usageEntries.
|
|
// aggregateUsage sums across those entries (one per agent.run()).
|
|
//
|
|
// caveat: if the agent promise rejected (timeout or uncaught throw) the
|
|
// usage was never pushed, so nothing gets persisted for that run. runs
|
|
// that returned AgentResult with success=false still report their partial
|
|
// usage because the harness populates AgentUsage before returning.
|
|
if (toolContext) {
|
|
const patch = aggregateUsage(toolState.usageEntries);
|
|
if (Object.keys(patch).length > 0) {
|
|
await patchWorkflowRunFields(toolContext, patch);
|
|
}
|
|
}
|
|
}
|
|
}
|