Files
Colin McDonnell 36ac64a5b6 fix(oss-codex): prefer user's uploaded Codex auth over OSS subsidy (#844)
* fix(oss-codex): prefer user's uploaded Codex auth over OSS subsidy

OSS-allowlisted repos with `CODEX_AUTH_JSON` uploaded via `pullfrog auth
codex` were still being routed through the OSS OpenRouter subsidy because
two paths ignored managed credentials:

- `hasProviderKey()` only checked `provider.envVars`, so an `openai/*`
  model with only `CODEX_AUTH_JSON` present silently fell back to
  `opencode/big-pickle` via `selectFallbackModelIfNeeded` — the maintainer
  saw "opencode/big-pickle (resolved from openai/gpt)" on CI even though
  Codex was configured.
- `run-context` set `proxyModel` for every OSS run unconditionally, which
  the action runtime threads through `payload.proxyModel` and uses to
  overwrite `OPENROUTER_API_KEY`. Even if `big-pickle` fallback hadn't
  fired, the runner would consume the $10 OSS subsidy key instead of the
  user's ChatGPT subscription.

Fix:

- Add `getModelAuthEnvVars()` covering both `envVars` and
  `managedCredentials` in `action/utils/apiKeys.ts`; route
  `hasProviderKey` + `validateAgentApiKey` through it.
- `run-context` now skips `proxyModel` for OSS repos when the configured
  model's provider has matching auth in Pullfrog-stored account/repo
  secrets, so the runner authenticates directly with the user's Codex
  subscription (or any other user-provided provider auth).

Triggered by mrlubos (`hey-api/openapi-ts`). Companion follow-up tracked
for the opaque "(no error message)" classifier swallow that masked the
OSS $10 cap exhaustion on PR #3872 runs 25815370844 + 25815443234.

* fix(oss): force Kimi K2 for OSS proxy + hide picker in console UI

OSS-funded runs were resolving `repo.model` through OpenRouter, so a
single Opus / GPT-5.5 run could burn an entire `oss_subsidy` key against
the per-key cap and crash mid-stream (e.g. `hey-api/openapi-ts` PR #3872
runs `25815370844` + `25815443234`, ~$9.20 each on a single key).

Force `DEFAULT_PROXY_MODEL` (Kimi K2.6 — ~10-50× cheaper) for every OSS
proxy mint, regardless of `repo.model`. Per-run spend stays bounded
within the cap by structure, not by hope. `repo.model` stays in the DB
unchanged — overriding at runtime means leaving the program restores the
user's prior pick without a migration.

UI: hide the model picker entirely on OSS repos in `AgentSettings`. The
field is effectively inert until the repo leaves the program, so
exposing it as if it were live was misleading. Replaced with a banner
naming Kimi K2 and pointing to `pullfrog auth …` as the opt-out path —
that lands the user on the existing #844 bug-2 branch (Pullfrog-stored
auth suppresses the OSS proxy entirely; runner uses user credentials
+ their preferred model).

ModelCostsInfo already has its own `isOss` branch for the cost copy, so
that section is unchanged.

* fix(oss): lowercase comment casing per AGENTS.md

* fix(oss): revert banner copy to 'It's on us.' framing per review

Maintainer felt 'Kimi K2' as the banner headline lost the warm 'we've
got you covered' framing that the existing OSS cost banner uses. Restore
'It's on us.' as the headline, move the model name into the body where
it explains the hardcoded choice and points to the opt-out (pullfrog
auth codex / account secret).

* docs(agents): screenshots must be of the live route, never synthetic

Caught myself building a temp `/dev/oss-ui-preview` route with hardcoded
JSX copy-pasted from the real component just to grab a screenshot — the
result told us nothing about whether the actual integrated UI worked,
and the user (rightly) called it out as a waste. Strengthen the rule:
screenshots must come from the live route in the running app, driven by
the actual component tree and real props. Note the GH OAuth interstitial
gotcha so the next agent gets through Clerk → GitHub sign-in on the
first try instead of bailing to a fake render. Also bans side-by-side
comparison screenshots unless explicitly requested.

* fix(oss): one 'It's on us.' banner, not two

OSS Agent settings was showing the message twice — once in the Model
section, once in the Model costs section right below it. Fold the cost
coverage into the model banner ('at no cost to you' + the spend stat)
and hide the Model costs subsection entirely for OSS. ModelCostsInfo
no longer needs `isOss` / `ossSpendThisMonthUsd` props — call site is
gated, so the OSS branch is dead. Removed it and the now-unused props.

Non-OSS rendering is unchanged: full Model picker + Model costs
subsection with Router / BYOK branches.

* feat(action): corepack-aware package manager provisioning before setup

customer setup scripts that did `npm i -g pnpm && pnpm install` were
installing whatever pnpm "latest" happens to be on the day the run fires,
not what the repo declares — and pnpm 11.3 silently writes a new
`packageManagerDependencies` block into lockfiles, which the agent's
"always push changes" rule then packages into a noisy PR (see #844).

resolve the project's pnpm/yarn pin from `package.json` (honoring pnpm
11+ precedence: `devEngines.packageManager` over `packageManager`) and
activate it via `corepack prepare ... --activate` BEFORE the setup hook
runs. corepack is bundled with node, so this is a no-op on managed
infra; failure (no corepack, no network, range-only version) degrades
to a warning and the existing PATH binary still runs.

also replaces the legacy `npm install -g <pm>@<v>` path in prep with
the same helper so behavior is consistent end-to-end. bun/deno still
use the legacy installer because corepack doesn't ship shims for them.

* chore(console): drop 'npm i -g pnpm' anti-pattern from setup-script placeholder

the suggested example trained customers to install pnpm unpinned, which
silently picks up whatever's latest at run time. that's exactly the
behavior #844 traced lockfile drift back to. now that prep handles
package-manager provisioning via corepack from the repo's declared pin,
the placeholder is just a frozen-lockfile install — load-bearing only
when the repo wants `pnpm install` to actually run (prep already does
that), but a much safer default for customers who do paste it in.

* refactor(action): introspect opencode models for BYOK detection

Replace the static `provider.envVars + provider.managedCredentials`
catalog gate in `selectFallbackModelIfNeeded` + `validateAgentApiKey`
with two `opencode models` captures around the auth merge:

  - `captureBaselineModels` BEFORE dbSecrets + Codex auth.json
  - `captureAuthorizedModels` AFTER both

The authorized set is the authoritative source for "can OpenCode route
this model" — strictly more accurate than the catalog, which can miss
new auth shapes (Codex was one, there will be more). The diff between
baseline and authorized is logged as `BYOK auth enabled N model(s)`
for operator visibility.

Sequencing changes in main.ts:

  - `createTempDirectory` hoisted out of the try block so
    `PULLFROG_TEMP_DIR` is set before the early opencode install
  - `agents.opencode.install()` + baseline capture before dbSecrets
  - `installCodexAuth()` hoisted up (idempotent — agent re-calls it
    inside run() and writes the same file)
  - authorized capture after Codex auth.json materializes
  - fallback + validateAgentApiKey receive the authorized set as a
    parameter; tests inject directly with no mocks

Deleted: `hasProviderKey`, `getModelAuthEnvVars`, `knownApiKeys` in
`action/utils/apiKeys.ts` (only `selectFallbackModelIfNeeded` consumed
them, and PR #844's catalog-extension fix is superseded by introspection).
`getModelEnvVars` / `getModelManagedCredentials` stay exported for UI
and the server-side OSS proxy heuristic in run-context/route.ts.

For the claude agent path, validateAgentApiKey keeps the static
single-provider check on `ANTHROPIC_API_KEY` / `CLAUDE_CODE_OAUTH_TOKEN`
— `opencode models` is opencode-specific. validateBedrockSetup /
validateVertexSetup also stay; they cover region/location/model-id
which `opencode models` doesn't catch.

When fallback engages, the post-fallback model is the guaranteed-free
`opencode/big-pickle`, so validateAgentApiKey is skipped — the fallback
gate already authoritatively decided "this model is OK to run".

* test(oss): temp add preview-844 to ossRepos for O4 e2e

* Revert "test(oss): temp add preview-844 to ossRepos for O4 e2e"

This reverts commit 8167e560126b2ac516c32ba1c63c36aa32ae4019.

* test(oss): temp add preview-844 to ossRepos for O5 e2e

* fix(action): skip validateAgentApiKey when proxyModel is set

The new opencode-models BYOK introspection in PR #844 captures the
authorized set BEFORE runProxyResolution mints OPENROUTER_API_KEY, so
the proxy slug (e.g. `openrouter/moonshotai/kimi-k2.6`) is never in
the set. validateAgentApiKey then spuriously threw "no API key found"
on every OSS run, even though the proxy key was minted correctly and
the inference would have worked.

Mirrors the analogous skip in `selectFallbackModelIfNeeded`: when
proxyModel is set, the server-side gate (`run-context/route.ts`) is
the authority and the proxy mint itself is the validation.

Caught by O5 e2e on `pullfrog/preview-844-heyapi-oss-bug`.

* Revert "test(oss): temp add preview-844 to ossRepos for O5 e2e"

This reverts commit 3bae075ceeb188ee272c45c13b5080e15bcd00a5.

* fix(action): discard hook-generated tracked-file drift before agent sees it

addresses bug 3 in #844: customer setup/post-checkout hooks like `pnpm install`
or `corepack prepare` left the working tree dirty (e.g. `M pnpm-lock.yaml`),
the agent took the prompt's "must push" rule literally, opened a spurious bot
PR for the lockfile drift, and we ate runs+spend on noise.

after each setup / post-checkout hook (opt-in via `normalizeWorkingTreeAfter`),
discard tracked-file mods with `git restore --staged --worktree .`. untracked
files are preserved — a hook that materializes a `.env` from a template, or
emits codegen output, stays visible to the agent.

guarded by a pre-hook `git status --porcelain` snapshot: if the tree was
already dirty before the hook ran (shouldn't happen — setup runs before any
working-tree writes; checkout_pr refuses to run dirty), we warn and skip the
discard rather than clobber whatever was there.

prepush hook (action/mcp/git.ts) intentionally does NOT opt in — its job is
to read the about-to-be-pushed state, not normalize it.

* test(oss): temp add preview-844 to ossRepos for bug 3 e2e (revert before merge)

* fix(action): skip eager pnpm/npm/etc install when no lockfile exists

second half of bug 3 in #844. the eager prep step assumed `pnpm install
--frozen-lockfile` (and equivalents) would fail cleanly without a lockfile,
leaving the tree untouched. that assumption is false for pnpm 11.1.1 against
a no-deps `package.json`: the command reports "Already up to date" with exit 0
AND silently materializes an empty `pnpm-lock.yaml` despite the `--frozen-lockfile`
flag. the resulting untracked file trips the post-run dirty-tree gate, the
agent reads it as "must push uncommitted work", and a spurious
"Add pnpm lockfile" PR lands. smoking gun: pullfrog/preview-844-heyapi-oss-bug
PRs #1/#2/#3, all auto-opened by the bot against a repo that contains nothing
but a one-line README + a no-deps package.json.

guard explicitly with an `existsSync` per manager. if the lockfile is absent,
skip eager prep entirely with an info log; the agent can install on demand
via the `setup` lifecycle hook (which non-frozen `pnpm install` would handle
correctly), or just leave deps uninstalled when the prompt doesn't need them
(e.g. the O5 "tell me a joke" path).

orthogonal to the lifecycle-hook normalization in 0051bd2a — together they
cover the full bug 3 surface:
  - eager prep can't materialize a lockfile (this commit)
  - setup/postCheckout hooks that rewrite tracked files have the drift
    discarded before the agent sees it (prior commit)

* fix(action): address Pullfrog review on hook normalization

two fixes in `executeLifecycleHook` from review on f6f3b32:

1. pre-hook snapshot was `git status --porcelain` which counts untracked
   files; in practice any repo with pre-existing untracked content (e.g.
   `.plans/`, an ignored-but-not-yet-gitignored scratch dir, codegen
   artifacts) would trip the guard and silently skip normalization,
   defeating the fix. switch to `git diff --name-only HEAD` so the gate
   measures the same thing the discard targets — tracked-file mods only.
   pre-existing untracked files are safe regardless because `git restore
   --staged --worktree .` never touches them.

2. normalization fired only on the happy path; a hook that updated a
   lockfile then exploded on a peer-dep conflict left tracked drift for
   the agent. move the call into a `finally` so it runs on success,
   non-zero exit, timeout, AND spawn failure. the pre-hook guard still
   protects pre-existing work in every case.

* Revert "test(oss): temp add preview-844 to ossRepos for bug 3 e2e (revert before merge)"

This reverts commit f6f3b325d6bf9a1720754ed1d39d248dab76cfa8.

* fix(action): use detect lockfile strategy for eager-prep gate

addresses Pullfrog review on be3c207b. two findings, one root cause:

- the hardcoded LOCKFILE_BY_MANAGER map missed `bun.lockb` and
  `npm-shrinkwrap.json`, two managers' accepted lockfile variants.
- `existsSync(join(cwd, lockfile))` only checked the immediate directory,
  breaking monorepo subpackages where the lockfile lives at the workspace
  root.

both fall out by replacing the custom check with `detect({ strategies:
["lockfile"] })`. the detector already walks up the tree (subpackage →
workspace root) and recognizes every accepted lockfile name across all
managers it supports. restricting to the `lockfile` strategy is load-
bearing: the default strategy set also matches on `packageManager` /
`devEngines.packageManager` package.json fields, which would return
non-null and re-mask the very case we're trying to detect (declared
manager, no lockfile committed — the O5 / hey-api preview repro).

drops the LOCKFILE_BY_MANAGER map entirely; no need for a second
detect() call since the existing one was only used for `agent`
resolution and that consumer is now after the lockfile gate, where
`detected` is guaranteed non-null.
2026-05-28 00:26:35 +00:00

180 lines
7.3 KiB
TypeScript

import { LIFECYCLE_HOOK_TIMEOUT_MS } from "../lifecycle.ts";
import { log } from "./cli.ts";
import {
SPAWN_ACTIVITY_TIMEOUT_CODE,
SPAWN_TIMEOUT_CODE,
SpawnTimeoutError,
spawn,
} from "./subprocess.ts";
export interface ExecuteLifecycleHookParams {
event: string;
script: string | null;
/**
* when true, after the hook runs (success or failure), discard tracked-file
* mods so the agent doesn't see hook-generated drift (e.g. `pnpm install`
* rewriting a lockfile). untracked files are preserved — hooks that
* intentionally materialize files (e.g. a `.env` from a template) stay
* visible to the agent. skipped (with a warning) if the tree had
* pre-existing tracked changes before the hook ran, so we never clobber
* pre-existing work; pre-existing untracked files are ignored for this
* gate because `git restore --staged --worktree .` doesn't touch them
* anyway. no-op when no script was configured.
*/
normalizeWorkingTreeAfter?: boolean;
}
/** structured failure info — `output` on the `exit` variant is trimmed
* stderr, falling back to stdout when stderr is empty. */
export type LifecycleHookFailure =
| { kind: "exit"; exitCode: number; output: string }
| { kind: "timeout" }
| { kind: "spawn"; spawnError: string };
export interface LifecycleHookResult {
/**
* human-readable warning when the hook failed. includes retry guidance:
* transient spawn/exit errors are worth retrying, timeouts and
* persistent failures are not. absent when the hook succeeded or was
* skipped. setup/post-checkout callers surface this verbatim; prepush
* builds its own message from `failure` instead.
*/
warning?: string;
/**
* structured failure info — undefined when the hook succeeded or was
* skipped. lets callers compose their own messaging without parsing the
* `warning` string.
*/
failure?: LifecycleHookFailure;
}
/**
* execute a lifecycle hook script if one is configured.
*
* soft-fails: instead of throwing on hook errors, returns a warning string
* (and structured failure info) so callers can choose whether to surface
* it (mcp tools) or upgrade it to a fatal error (setup). timeouts are
* flagged as non-retryable in the warning text.
*/
export async function executeLifecycleHook(
params: ExecuteLifecycleHookParams
): Promise<LifecycleHookResult> {
if (!params.script) return {};
log.info(`» executing ${params.event} lifecycle hook...`);
// snapshot tracked-file mods BEFORE the hook runs so we can distinguish
// hook-generated drift from pre-existing work. both hook windows should
// start clean in normal operation (setup runs before any working-tree
// writes; checkout_pr refuses to run with a dirty tree), but if that
// invariant breaks we'd rather warn than discard whatever was there.
// pre-existing untracked files don't matter here — `git restore --staged
// --worktree .` never touches untracked files, so they're never at risk.
const preHookTrackedCount = params.normalizeWorkingTreeAfter
? (await runGitLines(["diff", "--name-only", "HEAD"])).length
: 0;
// single try/finally so normalization fires on success AND failure paths.
// a hook that fails partway through (e.g. `pnpm install` updates the
// lockfile then explodes on a peer-dep conflict) leaves the same kind of
// drift a successful run does, and the agent will see it next regardless
// of which path we took. failure-mode messaging is unchanged; the only
// delta is that we don't return tracked drift to the agent.
let result: LifecycleHookResult;
try {
try {
const spawnResult = await spawn({
cmd: "bash",
args: ["-c", params.script],
env: process.env,
timeout: LIFECYCLE_HOOK_TIMEOUT_MS,
activityTimeout: 0,
onStdout: (chunk) => process.stdout.write(chunk),
onStderr: (chunk) => process.stderr.write(chunk),
});
if (spawnResult.exitCode !== 0) {
const output = (spawnResult.stderr || spawnResult.stdout).trim();
result = {
failure: { kind: "exit", output, exitCode: spawnResult.exitCode },
warning:
`lifecycle hook '${params.event}' failed with exit code ${spawnResult.exitCode}. ` +
`output: ${output || "(empty)"}. ` +
`retry the operation if the failure looks flaky (network blips, transient rate limits). ` +
`do NOT retry if the script is broken (missing commands, syntax errors) or the error is persistent.`,
};
} else {
log.info(`» ${params.event} lifecycle hook completed successfully`);
result = {};
}
} catch (err) {
const isTimeout =
err instanceof SpawnTimeoutError &&
(err.code === SPAWN_TIMEOUT_CODE || err.code === SPAWN_ACTIVITY_TIMEOUT_CODE);
if (isTimeout) {
const minutes = Math.round(LIFECYCLE_HOOK_TIMEOUT_MS / 60000);
result = {
failure: { kind: "timeout" },
warning:
`lifecycle hook '${params.event}' timed out after ${minutes}min. ` +
`do NOT retry — the script is likely hung or doing too much work. ` +
`ask the repo owner to simplify the hook (e.g. move long-running work out of the hook, add caching, or split it).`,
};
} else {
const msg = err instanceof Error ? err.message : String(err);
result = {
failure: { kind: "spawn", spawnError: msg },
warning:
`lifecycle hook '${params.event}' failed to spawn: ${msg}. ` +
`this is likely a transient failure — retry the operation.`,
};
}
}
} finally {
if (params.normalizeWorkingTreeAfter) {
await normalizeWorkingTreeAfterHook({ event: params.event, preHookTrackedCount });
}
}
return result;
}
/**
* discard tracked-file mods left by a lifecycle hook so the agent's next
* `git status` matches the pre-hook state. untracked files (e.g. a `.env`
* the hook materialized from a template) are left alone — the agent decides
* what to do with them. skipped (with a warning) when the tree had
* pre-existing tracked changes before the hook ran, so pre-existing work
* is never clobbered. idempotent: a second call on a clean tree is a no-op
* and stays quiet.
*/
async function normalizeWorkingTreeAfterHook(params: {
event: string;
preHookTrackedCount: number;
}): Promise<void> {
if (params.preHookTrackedCount > 0) {
log.warning(
`» working tree had ${params.preHookTrackedCount} pre-existing tracked changes before ${params.event} hook; ` +
`skipping post-hook normalization to avoid clobbering pre-existing work`
);
return;
}
const trackedCount = (await runGitLines(["diff", "--name-only", "HEAD"])).length;
if (trackedCount === 0) return;
await runGit(["restore", "--staged", "--worktree", "."]);
log.info(`» discarded ${trackedCount} tracked changes from ${params.event} hook`);
}
async function runGit(args: string[]): Promise<string> {
const result = await spawn({ cmd: "git", args, env: process.env, activityTimeout: 0 });
if (result.exitCode !== 0) {
throw new Error(
`git ${args.join(" ")} failed (exit ${result.exitCode}): ${result.stderr.trim() || "(no stderr)"}`
);
}
return result.stdout;
}
async function runGitLines(args: string[]): Promise<string[]> {
return (await runGit(args)).split("\n").filter(Boolean);
}