* fix(oss-codex): prefer user's uploaded Codex auth over OSS subsidy
OSS-allowlisted repos with `CODEX_AUTH_JSON` uploaded via `pullfrog auth
codex` were still being routed through the OSS OpenRouter subsidy because
two paths ignored managed credentials:
- `hasProviderKey()` only checked `provider.envVars`, so an `openai/*`
model with only `CODEX_AUTH_JSON` present silently fell back to
`opencode/big-pickle` via `selectFallbackModelIfNeeded` — the maintainer
saw "opencode/big-pickle (resolved from openai/gpt)" on CI even though
Codex was configured.
- `run-context` set `proxyModel` for every OSS run unconditionally, which
the action runtime threads through `payload.proxyModel` and uses to
overwrite `OPENROUTER_API_KEY`. Even if `big-pickle` fallback hadn't
fired, the runner would consume the $10 OSS subsidy key instead of the
user's ChatGPT subscription.
Fix:
- Add `getModelAuthEnvVars()` covering both `envVars` and
`managedCredentials` in `action/utils/apiKeys.ts`; route
`hasProviderKey` + `validateAgentApiKey` through it.
- `run-context` now skips `proxyModel` for OSS repos when the configured
model's provider has matching auth in Pullfrog-stored account/repo
secrets, so the runner authenticates directly with the user's Codex
subscription (or any other user-provided provider auth).
Triggered by mrlubos (`hey-api/openapi-ts`). Companion follow-up tracked
for the opaque "(no error message)" classifier swallow that masked the
OSS $10 cap exhaustion on PR #3872 runs 25815370844 + 25815443234.
* fix(oss): force Kimi K2 for OSS proxy + hide picker in console UI
OSS-funded runs were resolving `repo.model` through OpenRouter, so a
single Opus / GPT-5.5 run could burn an entire `oss_subsidy` key against
the per-key cap and crash mid-stream (e.g. `hey-api/openapi-ts` PR #3872
runs `25815370844` + `25815443234`, ~$9.20 each on a single key).
Force `DEFAULT_PROXY_MODEL` (Kimi K2.6 — ~10-50× cheaper) for every OSS
proxy mint, regardless of `repo.model`. Per-run spend stays bounded
within the cap by structure, not by hope. `repo.model` stays in the DB
unchanged — overriding at runtime means leaving the program restores the
user's prior pick without a migration.
UI: hide the model picker entirely on OSS repos in `AgentSettings`. The
field is effectively inert until the repo leaves the program, so
exposing it as if it were live was misleading. Replaced with a banner
naming Kimi K2 and pointing to `pullfrog auth …` as the opt-out path —
that lands the user on the existing #844 bug-2 branch (Pullfrog-stored
auth suppresses the OSS proxy entirely; runner uses user credentials
+ their preferred model).
ModelCostsInfo already has its own `isOss` branch for the cost copy, so
that section is unchanged.
* fix(oss): lowercase comment casing per AGENTS.md
* fix(oss): revert banner copy to 'It's on us.' framing per review
Maintainer felt 'Kimi K2' as the banner headline lost the warm 'we've
got you covered' framing that the existing OSS cost banner uses. Restore
'It's on us.' as the headline, move the model name into the body where
it explains the hardcoded choice and points to the opt-out (pullfrog
auth codex / account secret).
* docs(agents): screenshots must be of the live route, never synthetic
Caught myself building a temp `/dev/oss-ui-preview` route with hardcoded
JSX copy-pasted from the real component just to grab a screenshot — the
result told us nothing about whether the actual integrated UI worked,
and the user (rightly) called it out as a waste. Strengthen the rule:
screenshots must come from the live route in the running app, driven by
the actual component tree and real props. Note the GH OAuth interstitial
gotcha so the next agent gets through Clerk → GitHub sign-in on the
first try instead of bailing to a fake render. Also bans side-by-side
comparison screenshots unless explicitly requested.
* fix(oss): one 'It's on us.' banner, not two
OSS Agent settings was showing the message twice — once in the Model
section, once in the Model costs section right below it. Fold the cost
coverage into the model banner ('at no cost to you' + the spend stat)
and hide the Model costs subsection entirely for OSS. ModelCostsInfo
no longer needs `isOss` / `ossSpendThisMonthUsd` props — call site is
gated, so the OSS branch is dead. Removed it and the now-unused props.
Non-OSS rendering is unchanged: full Model picker + Model costs
subsection with Router / BYOK branches.
* feat(action): corepack-aware package manager provisioning before setup
customer setup scripts that did `npm i -g pnpm && pnpm install` were
installing whatever pnpm "latest" happens to be on the day the run fires,
not what the repo declares — and pnpm 11.3 silently writes a new
`packageManagerDependencies` block into lockfiles, which the agent's
"always push changes" rule then packages into a noisy PR (see #844).
resolve the project's pnpm/yarn pin from `package.json` (honoring pnpm
11+ precedence: `devEngines.packageManager` over `packageManager`) and
activate it via `corepack prepare ... --activate` BEFORE the setup hook
runs. corepack is bundled with node, so this is a no-op on managed
infra; failure (no corepack, no network, range-only version) degrades
to a warning and the existing PATH binary still runs.
also replaces the legacy `npm install -g <pm>@<v>` path in prep with
the same helper so behavior is consistent end-to-end. bun/deno still
use the legacy installer because corepack doesn't ship shims for them.
* chore(console): drop 'npm i -g pnpm' anti-pattern from setup-script placeholder
the suggested example trained customers to install pnpm unpinned, which
silently picks up whatever's latest at run time. that's exactly the
behavior #844 traced lockfile drift back to. now that prep handles
package-manager provisioning via corepack from the repo's declared pin,
the placeholder is just a frozen-lockfile install — load-bearing only
when the repo wants `pnpm install` to actually run (prep already does
that), but a much safer default for customers who do paste it in.
* refactor(action): introspect opencode models for BYOK detection
Replace the static `provider.envVars + provider.managedCredentials`
catalog gate in `selectFallbackModelIfNeeded` + `validateAgentApiKey`
with two `opencode models` captures around the auth merge:
- `captureBaselineModels` BEFORE dbSecrets + Codex auth.json
- `captureAuthorizedModels` AFTER both
The authorized set is the authoritative source for "can OpenCode route
this model" — strictly more accurate than the catalog, which can miss
new auth shapes (Codex was one, there will be more). The diff between
baseline and authorized is logged as `BYOK auth enabled N model(s)`
for operator visibility.
Sequencing changes in main.ts:
- `createTempDirectory` hoisted out of the try block so
`PULLFROG_TEMP_DIR` is set before the early opencode install
- `agents.opencode.install()` + baseline capture before dbSecrets
- `installCodexAuth()` hoisted up (idempotent — agent re-calls it
inside run() and writes the same file)
- authorized capture after Codex auth.json materializes
- fallback + validateAgentApiKey receive the authorized set as a
parameter; tests inject directly with no mocks
Deleted: `hasProviderKey`, `getModelAuthEnvVars`, `knownApiKeys` in
`action/utils/apiKeys.ts` (only `selectFallbackModelIfNeeded` consumed
them, and PR #844's catalog-extension fix is superseded by introspection).
`getModelEnvVars` / `getModelManagedCredentials` stay exported for UI
and the server-side OSS proxy heuristic in run-context/route.ts.
For the claude agent path, validateAgentApiKey keeps the static
single-provider check on `ANTHROPIC_API_KEY` / `CLAUDE_CODE_OAUTH_TOKEN`
— `opencode models` is opencode-specific. validateBedrockSetup /
validateVertexSetup also stay; they cover region/location/model-id
which `opencode models` doesn't catch.
When fallback engages, the post-fallback model is the guaranteed-free
`opencode/big-pickle`, so validateAgentApiKey is skipped — the fallback
gate already authoritatively decided "this model is OK to run".
* test(oss): temp add preview-844 to ossRepos for O4 e2e
* Revert "test(oss): temp add preview-844 to ossRepos for O4 e2e"
This reverts commit 8167e560126b2ac516c32ba1c63c36aa32ae4019.
* test(oss): temp add preview-844 to ossRepos for O5 e2e
* fix(action): skip validateAgentApiKey when proxyModel is set
The new opencode-models BYOK introspection in PR #844 captures the
authorized set BEFORE runProxyResolution mints OPENROUTER_API_KEY, so
the proxy slug (e.g. `openrouter/moonshotai/kimi-k2.6`) is never in
the set. validateAgentApiKey then spuriously threw "no API key found"
on every OSS run, even though the proxy key was minted correctly and
the inference would have worked.
Mirrors the analogous skip in `selectFallbackModelIfNeeded`: when
proxyModel is set, the server-side gate (`run-context/route.ts`) is
the authority and the proxy mint itself is the validation.
Caught by O5 e2e on `pullfrog/preview-844-heyapi-oss-bug`.
* Revert "test(oss): temp add preview-844 to ossRepos for O5 e2e"
This reverts commit 3bae075ceeb188ee272c45c13b5080e15bcd00a5.
* fix(action): discard hook-generated tracked-file drift before agent sees it
addresses bug 3 in #844: customer setup/post-checkout hooks like `pnpm install`
or `corepack prepare` left the working tree dirty (e.g. `M pnpm-lock.yaml`),
the agent took the prompt's "must push" rule literally, opened a spurious bot
PR for the lockfile drift, and we ate runs+spend on noise.
after each setup / post-checkout hook (opt-in via `normalizeWorkingTreeAfter`),
discard tracked-file mods with `git restore --staged --worktree .`. untracked
files are preserved — a hook that materializes a `.env` from a template, or
emits codegen output, stays visible to the agent.
guarded by a pre-hook `git status --porcelain` snapshot: if the tree was
already dirty before the hook ran (shouldn't happen — setup runs before any
working-tree writes; checkout_pr refuses to run dirty), we warn and skip the
discard rather than clobber whatever was there.
prepush hook (action/mcp/git.ts) intentionally does NOT opt in — its job is
to read the about-to-be-pushed state, not normalize it.
* test(oss): temp add preview-844 to ossRepos for bug 3 e2e (revert before merge)
* fix(action): skip eager pnpm/npm/etc install when no lockfile exists
second half of bug 3 in #844. the eager prep step assumed `pnpm install
--frozen-lockfile` (and equivalents) would fail cleanly without a lockfile,
leaving the tree untouched. that assumption is false for pnpm 11.1.1 against
a no-deps `package.json`: the command reports "Already up to date" with exit 0
AND silently materializes an empty `pnpm-lock.yaml` despite the `--frozen-lockfile`
flag. the resulting untracked file trips the post-run dirty-tree gate, the
agent reads it as "must push uncommitted work", and a spurious
"Add pnpm lockfile" PR lands. smoking gun: pullfrog/preview-844-heyapi-oss-bug
PRs #1/#2/#3, all auto-opened by the bot against a repo that contains nothing
but a one-line README + a no-deps package.json.
guard explicitly with an `existsSync` per manager. if the lockfile is absent,
skip eager prep entirely with an info log; the agent can install on demand
via the `setup` lifecycle hook (which non-frozen `pnpm install` would handle
correctly), or just leave deps uninstalled when the prompt doesn't need them
(e.g. the O5 "tell me a joke" path).
orthogonal to the lifecycle-hook normalization in 0051bd2a — together they
cover the full bug 3 surface:
- eager prep can't materialize a lockfile (this commit)
- setup/postCheckout hooks that rewrite tracked files have the drift
discarded before the agent sees it (prior commit)
* fix(action): address Pullfrog review on hook normalization
two fixes in `executeLifecycleHook` from review on f6f3b32:
1. pre-hook snapshot was `git status --porcelain` which counts untracked
files; in practice any repo with pre-existing untracked content (e.g.
`.plans/`, an ignored-but-not-yet-gitignored scratch dir, codegen
artifacts) would trip the guard and silently skip normalization,
defeating the fix. switch to `git diff --name-only HEAD` so the gate
measures the same thing the discard targets — tracked-file mods only.
pre-existing untracked files are safe regardless because `git restore
--staged --worktree .` never touches them.
2. normalization fired only on the happy path; a hook that updated a
lockfile then exploded on a peer-dep conflict left tracked drift for
the agent. move the call into a `finally` so it runs on success,
non-zero exit, timeout, AND spawn failure. the pre-hook guard still
protects pre-existing work in every case.
* Revert "test(oss): temp add preview-844 to ossRepos for bug 3 e2e (revert before merge)"
This reverts commit f6f3b325d6bf9a1720754ed1d39d248dab76cfa8.
* fix(action): use detect lockfile strategy for eager-prep gate
addresses Pullfrog review on be3c207b. two findings, one root cause:
- the hardcoded LOCKFILE_BY_MANAGER map missed `bun.lockb` and
`npm-shrinkwrap.json`, two managers' accepted lockfile variants.
- `existsSync(join(cwd, lockfile))` only checked the immediate directory,
breaking monorepo subpackages where the lockfile lives at the workspace
root.
both fall out by replacing the custom check with `detect({ strategies:
["lockfile"] })`. the detector already walks up the tree (subpackage →
workspace root) and recognizes every accepted lockfile name across all
managers it supports. restricting to the `lockfile` strategy is load-
bearing: the default strategy set also matches on `packageManager` /
`devEngines.packageManager` package.json fields, which would return
non-null and re-mask the very case we're trying to detect (declared
manager, no lockfile committed — the O5 / hey-api preview repro).
drops the LOCKFILE_BY_MANAGER map entirely; no need for a second
detect() call since the existing one was only used for `agent`
resolution and that consumer is now after the lockfile gate, where
`detected` is guaranteed non-null.
reflection turn is a meta-turn for editing the learnings file; the
task's `result` output was already finalized on the previous turn.
gemini pro re-triggers on the standing "call set_output when done"
system instruction during reflection and clobbers the value with the
literal word "done" (see ci run 26529624199, smoke test on
providers-live google/gemini-pro). add an explicit prohibition to the
reflection prompt; the snapshot/restore in runPostRunRetryLoop
remains as defense in depth.
models occasionally call `pullfrog_git({command:"status", args:["status"]})`,
which shells out to `git status status`. git silently treats args[0] as a
pathspec — when no file/dir matches, status prints "nothing to commit,
working tree clean" even on a dirty tree. observed in production
(Skn0tt/beckerbuch run 26519563044): the agent looped trying to reconcile
that against a real diff, burned ~$3 / ~6min of opus, and only escaped when
it switched to `args: ["--porcelain"]`. generalises to every subcommand
(`diff diff`, `log log`, ...).
guard `args[0]?.toLowerCase() === command.toLowerCase()` with a directed
throw pointing the model at the disambiguated `args: ["--", "<name>"]`
escape hatch for the rare legitimate pathspec case (`--` works under every
subcommand, unlike a bare positional which can be parsed as a ref by
log/diff/checkout/restore/reset).
description also leads with the no-args case and explicitly forbids
repeating the subcommand in args. schema already had args.optional().
also includes in-flight working-tree work:
- postRun: snapshot/restore toolState.output across reflection turn so reflection prompt can't clobber task-turn output (gemini pro regression)
- toolState: widen `output` to `string | undefined` for assignability
- uninstallFeedback: suspend-mode emails now CTA the GitHub unsuspend page when accountType is known; delete events keep console pointer
* fix(askpass): scope code + script lifetime to one $git() call, not first password prompt
LFS pre-push (and any auth-bound sibling subprocess) consumed the single-use
code AND triggered the script self-delete, so git's own push call then hit
`fatal: cannot exec '/tmp/pullfrog-…/askpass-…js'` and our server treated
the legitimate retry as tamper, revoking the installation token. Observed
on nteract/nteract#2987 (LFS repo).
`gitAuthServer` codes are now `active` until `$git()`'s finally calls
`revoke()`; the script no longer self-deletes (finally already unlinks).
Replay after revoke still trips 409 + token revocation, which is the
realistic exfiltration vector we care about.
* fix(askpass): drop wall-clock TTL on active codes
Copilot review on #841 noticed the 5-minute CODE_TTL_MS still applied to
active codes, which would re-introduce the original LFS failure mode at a
different boundary: a large LFS push lasting >5min would hit a 404 mid-
call. $git() uses `activityTimeout: 0` precisely because git fetch/push
can take arbitrarily long, so any wall-clock TTL on active codes is wrong.
Active codes now live until revoke() is called (in $git()'s finally) or
the auth server is closed. Revoked codes keep their 60s replay trap.
* docs(askpass): purge stale single-use vocabulary; align error message
Pullfrog review on #841 surfaced four doc-drift sites that still described
the pre-PR single-use model:
- wiki/security.md — 3 references (overview prose, bullets, threat-mitigation table)
- action/utils/gitAuth.ts — file-level JSDoc + the 409 error message
- wiki/askpass.md — error message quoted in the tamper-evident section
- action/utils/gitAuthServer.ts — per-prompt invocation comment was ambiguous
All updated to match the active|revoked vocabulary; error message is now
"askpass code was replayed after revoke, token revoked".
* fix: 6 unaddressed log-audit / run-audit findings
- #836 + #818 (clerk middleware SyntaxError on action-runtime endpoints):
narrow proxy.ts matcher to exclude /api/repo/<owner>/<repo>/run-context,
/api/runtime/*, /api/proxy-token. these are server-to-server with their
own auth and have no Clerk session to evaluate, so clerkMiddleware's
decodeJwt throws turn into 500s on every request.
- #837 (npx EBADDEVENGINES on customer's package.json): change runCli's
bootstrap cwd from $GITHUB_WORKSPACE to os.tmpdir() so npm v11+ doesn't
enforce devEngines.packageManager from the customer's tree before our
bootstrap can install pullfrog. CLI process.chdir's to payload.cwd
internally, so the runtime work still happens in $GITHUB_WORKSPACE.
- #838 (createWorkflowDispatch silently dropped 39 user runs on a 5xx
spike): add bounded retry (3 attempts, ~750ms total) on Octokit 5xx and
network errors inside dispatchReservedRun. preserves the existing 422
"Unexpected inputs" path. retry budget stays well under GitHub's 10s
webhook redelivery window.
- #833 (bail() redirect("/signout") propagated NEXT_REDIRECT into webhook
handlers, 40+ 500s/24h): drop the redirect side-effect; bail now just
classifies bad-credentials as non-retryable and propagates. UI flows
that wanted auto-signout on revoked tokens can detect it themselves;
the side-effect was wrong for any non-page caller.
- #835 (BYOK provider billing-exhausted fell through to raw error
renderer, 37 review-mode runs/24h with no PR-side signal):
- extend providerErrors patterns with Anthropic "credit balance is too
low" + extract isProviderBillingExhausted / extractProviderId helpers
- add a renderer branch in runErrorRenderer.ts that names the provider
(parsed from providerID=) and links to its billing dashboard
- route handleAgentResult's !result.success path through the renderer
+ reportErrorToComment with createIfMissing, so review-mode and
silent triggers get an actionable PR comment regardless of mode
- #834 (post-hook ERR_MODULE_NOT_FOUND already fixed on main, hardening):
- add a vitest invariant that walks the entryPost.ts import graph and
refuses any non-relative / non-node: specifier — catches the next
`@actions/core` slip-up before publish
- add an analyze-logs classifier so future entryPost crashes surface as
failure:post-hook-module-not-found instead of hiding inside
failure:unknown / failure:git-lock-file
Co-authored-by: Cursor <cursoragent@cursor.com>
* anneal: fix dead-code matcher, 404 url, silent-trigger gate, and grammar regression
Round-1 anneal pass on the audit-fixes PR surfaced two critical issues +
several majors that the original fixes shipped with:
- proxy.ts matcher 1 (`(?!_next|[^?]*\.(...))`) still caught every
/api/... route because matcher arrays are OR'd. The narrowing in
matcher 2 was dead code → middleware still 500'd on /api/runtime/*,
/api/proxy-token, /api/repo/.../run-context. Carve-out now lives in
BOTH matchers.
- opencode.ai/billing returns 404; canonical top-up surface is /zen.
deepseek /usage is the consumption page, not the top-up flow (/top_up
is correct). google /apikey is the keys list, not billing (/usage is
the spend dashboard). All three URL strings updated.
- handleAgentResult gated reportErrorToComment behind `if (!ctx.silent)`,
contradicting the createIfMissing intent — silent IncrementalReview /
pull_request_synchronize / auto-label still got zero PR signal on BYOK
billing exhaustion, the exact failure mode #835 was meant to fix.
Moved createIfMissing into finalizeSuccessRun's existing render-and-
post block (single source of truth), reverted handleAgentResult to its
prior shape. Side benefit: drops the double-PATCH that fired on every
non-silent !success path with an existing progress comment.
- Anthropic-direct error rendered "**Your your provider account is out
of credit.**" because Anthropic SDK has no providerID= tag, so
extractProviderId returned null and the headline composed
"Your " + "your provider". Added detectProviderId Anthropic fallback
(matches "Anthropic API" / "credit balance is too low") so the link
is reachable AND made the headline conditional on whether a provider
id was detected.
- Reordered renderRunError classifier: BYOK billing-exhausted now runs
BEFORE api-key auth detection. Providers commonly return 401 for
billing exhaustion (DeepSeek, Gemini), and the OpenCode harness logs
often include "API Error: 401" in the raw error body, which
isApiKeyAuthError would otherwise match — surfacing "rotate your key"
when the actual fix is "top up credits".
- isTransientUpstreamError missed ENOTFOUND / ENETUNREACH / EHOSTUNREACH
(undici DNS-class failures Octokit doesn't wrap with a status). Added
to the prefix alternation.
- Tightened "10s webhook redelivery budget" / "GitHub redelivers"
wording in triggerWorkflow.ts and bail.ts JSDoc — GitHub's 10s is the
response timeout (it doesn't auto-redeliver); upstream webhook proxy
retries are what multiplied the failure.
Co-authored-by: Cursor <cursoragent@cursor.com>
* anneal r2: unbreak proxy.ts matcher 2, extend carve-out, mkdtemp bootstrap
Round-2 anneal (security + cross-cutting lenses) on top of the round-1
audit-fixes commit caught a critical regression + several majors:
- proxy.ts matcher 2 from r1 (`/(api|trpc)(?!...)(.*)`) does NOT compile.
path-to-regexp rejects a top-level `(?!` after `)` as "Pattern cannot
start with '?'", and Next.js's SourceSchema runs the same validator at
build time and aborts via `process.exit(1)`. PR #840's Vercel + preview
deployments have been failing since 978eca26 for exactly this reason.
Fixed by nesting the lookahead inside an outer parameter group:
`/(api|trpc)((?!...).*)`. Same shape matcher 1 already uses, which is
why m1 always compiled. Verified `pnpm next build` succeeds end-to-end.
- proxy.ts carve-out was incomplete relative to its own justification.
The "no Clerk session, decodeJwt 500" failure mode applies to ALL
server-to-server action-runtime endpoints — five more share the exact
shape: /api/repo/.../learnings, /api/repo/.../pr/.../summary-comment,
/api/repo/.../issue/.../plan-comment, /api/workflow-run/, and
/api/github/installation-token. Extended both matchers. /api/upload/
signed-url stays in (dual auth: Clerk session OR bearer JWT — needs
middleware for the user path).
- proxy.ts carve-outs were unanchored: a future /api/proxy-token-info,
/api/proxy-tokens, or /api/repo/X/Y/run-context-foo would silently
bypass Clerk. Added `(?:$|/)` for path-prefix carve-outs (allow exact
match or sub-path), `$` for routes with browser-callable siblings
(e.g. `learnings/history` is browser-Clerk, `learnings$` is action-
bearer-JWT). Verified 30/31 routes via path-to-regexp test harness.
- runCli.ts cwd flipped from $GITHUB_WORKSPACE to os.tmpdir() in r1
(#837 fix for npm v11 devEngines.packageManager EBADDEVENGINES). But
$TMPDIR is overridable from a prior $GITHUB_ENV step — a customer-
authored or compromised prior step can plant /atk/node_modules/
pullfrog/ and `echo "TMPDIR=/atk" >> $GITHUB_ENV`, and our npx
--yes pullfrog@<v> bootstrap resolves the local install first,
executing attacker code with full action env (provider keys, OIDC,
installation token, CODEX_AUTH_JSON). Switched to mkdtempSync(join
(tmpdir(), "pullfrog-bootstrap-")) — fresh per-invocation 0700 dir,
not pre-writable by anything earlier in the job.
- runLifecycle.ts: writeRunErrorOutputs (catch-path) didn't pass
createIfMissing: true, contradicting the symmetric intent of the
r1 finalizeSuccessRun fix. Silent triggers (IncrementalReview /
pull_request_synchronize / auto-label) that throw past the success
path still got zero PR signal — exact failure mode #835 was meant
to close. Now both paths pass createIfMissing: true.
- runErrorRenderer.ts detectProviderId regex `/Anthropic API|credit
balance is too low/i` could mis-tag a non-Anthropic billing-exhausted
error that mentioned "Anthropic API" in passing (fallback-chain agent
prompt text, OpenCode harness logs). Tightened to /credit balance is
too low/i — Anthropic-specific phrasing, sufficient for the direct-
Anthropic SDK case the fallback exists to handle.
- runErrorRenderer.ts JSDoc: r1's classifier reorder put hang at #6 in
code but the JSDoc still listed it at #2. Reordered the doc to match
dispatch order, with explicit note that hang is a sub-source for the
api-key check (which is why hangBody is precomputed early).
- analyze-logs.ts: ERR_MODULE_NOT_FOUND.*entryPost regex needs `s`
flag so it survives Node v23+ stack-trace reformatting onto multiple
lines. One-char fix to defend the #834 classification bucket.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
- utils/ossRepos.ts: add middleapi/orpc (orpc.dev, 5.2k stars) to the
oss allowlist so any future install mints via mintOssKey ($10 cap,
pullfrog absorbs).
- scripts/skill.ts + pnpm skill: scaffold .agents/skills/<name>/SKILL.md
+ .claude/skills/<name> symlink. patch + skill skills written using it.
- AGENTS.md: hard-ban vi.* mocking apis; document pnpm skill workflow.
- audit follow-through: drop pure-mock test files (action/utils/lifecycle,
action/utils/timer, test/handleNoInstall) and trim action/mcp/review to
the non-mock cases.
- wiki/scripts.md: row for scripts/skill.ts.
c45a07e5 swapped the opencode harness default to moonshotai/kimi-k2,
which broke push-enabled (Kimi reported delete_branch as "auth failed"
where sonnet handles it). The agnostic test matrix expects Sonnet-grade
tool-calling; switching to Kimi was scope creep on top of the original
"don't default to opus" ask — opencode's default was already sonnet, not
opus, so nothing needed to change there.
* expose push: restricted to users
the action runtime already understood `disabled | restricted | enabled`
for `push` (see action/external.ts, action/mcp/git.ts); the DB enum, the
zod schema, the React settings UI, and the action.yml input docs lagged
behind. catches all four up. UI surfaces a 2-state toggle (restricted
↔ enabled) in the Security card alongside shell isolation — `disabled`
stays in the enum so non-UI callers (action.yml input) still accept it
but is intentionally hidden from the console.
* address reviews: doc full restricted scope + disabled UI edge case
reviewers flagged that "restricted" blocks more than default-branch
pushes (also delete_branch + push_tags per action/mcp/git.ts:621-700),
and that the popover's "off = full push access" claim is wrong for
repos whose workflow sets `push: disabled` (also renders as OFF since
the toggle is `props.push === "restricted"`). action.yml description
now enumerates the full restricted scope, and the popover frames off
behavior in terms of the workflow input so disabled repos aren't
misrepresented.
- opencode test default: claude-sonnet-4-6 → moonshotai/kimi-k2. claude-code
default stays on sonnet (the agent-under-test for that path is Claude
Code); the opencode path doesn't need an Anthropic model.
- byok-no-keys-fallback fixture: anthropic/claude-opus → moonshotai/kimi-k2.
test verifies fallback to opencode/big-pickle, so the configured model
is never invoked; picking a non-opus alias avoids burning credits if
the fallback ever regresses.
- BEDROCK_MODEL_ID in both test workflows: us.anthropic.claude-opus-4-6-v1
→ us.anthropic.claude-sonnet-4-6. env var is required by the ci.test
invariant that every provider's env vars are wired into the workflow.
* adhoc: push:restricted adversarial pentest
enumerates the 16 attack vectors the deep audit identified as load-bearing
for `push: restricted`. used to drive e2e verification against the preview
repo's pullfrog.yml; also runnable via pnpm runtest locally.
validator only asserts that the repo's default branch SHA didn't move —
the per-attack outputs are the deliverable for human review (the test
exists to feed adversarial runs, not to be a CI guard).
* wipe runner leak surface before agent spawn
the GHA runner persists credentials inside $RUNNER_TEMP that an MCP-shell
agent can grep — _runner_file_commands/set_output_* (from any composite
step that called core.setOutput, e.g. pullfrog/pullfrog/get-installation-token
which leaks a ghs_… installation token), <uuid>.sh rendered step scripts
(whose run: | body embeds ${{ ... }} expressions literally before write),
and git-credentials-*.config from actions/checkout@v6.
snapshot-and-delete that surface at action startup, after our own token is
in memory and before setupGit. preserves $GITHUB_OUTPUT, $GITHUB_ENV, and
$GITHUB_STATE so pullfrog's result output and post: hook still work.
setupGit's existing removeIncludeIfEntries call strips the matching
dangling includeIf.gitdir:....path entries from the user's .git/config.
does not tighten isGitCommand — that's a UX guard, not a security
boundary, and trivially bypassable via bash -c, absolute paths, symlinks,
python subprocess. the security boundary is the absence of credentials on
disk for those bypassed shells to authenticate with.
verified end-to-end by re-firing action/test/adhoc/pushRestrictedAdversarial
against pullfrog/preview-827-push-restricted-pentest.
* preserve all runner file-command paths from wipe
addresses pullfrog review on f7f5143b: GITHUB_STEP_SUMMARY also lives at
$RUNNER_TEMP/_runner_file_commands/step_summary_<uuid> and is read by the
runner AFTER our step exits to render the job summary in the GH UI. wiping
it silently broke pullfrog's job summary output. preserve GITHUB_PATH too
for symmetry — it's the same allocation pattern, and a step or post hook
that appends a directory expects the file to exist.
set of file-command env vars enumerated in @actions/core:
GITHUB_ENV, GITHUB_OUTPUT, GITHUB_PATH, GITHUB_STATE, GITHUB_STEP_SUMMARY
the skills CLI rejects "claude" — its valid list is claude-code,
opencode, cursor, etc. caused agent-browser skill install to fail
on every claude run.
* surface agent failures in job summary (#632)
when the agent harness returns `{success: false, error}`, main.ts went
through the success path so the catch block — which renders the
`### ❌ Pullfrog failed` banner via renderRunError — never fired. result:
the GitHub Actions job summary showed only the partial body + usage
table, no error block. the progress comment had a narrow workaround
that re-implemented the api-key classifier inline.
unify: in finalizeSuccessRun, call renderRunError once when `!success`,
use `.summary` for the job summary (prepended to the existing
body/usage parts) and `.comment` for the progress comment. removes the
duplicated isApiKeyAuthError / formatApiKeyErrorSummary branch and
picks up BillingError reclassification + hang body for free.
* docs: note dual-surface failure rendering in finalizeSuccessRun
* fix copilot review nit: clarify which renderRunError classifications carry the H3 banner
* checkout_pr: refuse unconditionally on dirty working tree
drop the live-HEAD comparison from the guard introduced in #796. any
checkout_pr call with staged or unstaged changes now throws, even when
HEAD is already on pr-N. no stashing, no idempotent escape hatch.
motivation is the zed-industries/cloud (2026-05-18) incident: shared-cwd
subagents make "carry edits along" semantics dangerous, and the
HEAD-equality predicate let a re-checkout silently inherit working-tree
state from a sibling agent. forcing commit/discard before any
PR-context operation eliminates the entire carry-forward failure class.
error names the PR number, lists dirty paths, and tells the agent to
commit/push/restore/clean before retrying.
* improve dirty-tree error: precise discard commands
copilot caught two sloppy bits in the error string:
- "push" alone does not clean a dirty tree (needs commit first)
- bare `git clean` is a no-op without `-fd`
reword to "commit (then push if needed), or discard with
`git restore --staged --worktree .` / `git clean -fd`" so the
guidance is actually actionable.
* checkout_pr: initial-branch invariant
setupGit captures `toolState.initialBranch` at run start via live
`git rev-parse --abbrev-ref HEAD`. checkout_pr refuses unless current
HEAD matches the run-entry branch or the target `pr-N` (idempotent
same-PR re-checkout). uses live rev-parse, not toolState.issueNumber
(poisonable per the PR #796 review).
refusal error names the current branch, target PR, recovery path
(`git checkout <initialBranch>` with the literal branch name), and
explicitly states routing around via the `git` tool is not sanctioned.
closes the zed-industries/cloud (2026-05-18) shape where a subagent
parked HEAD on someone else's `pr-X` and the orchestrator's next
checkout_pr inherited that position.
* reviewfrog: enforce canonical diff + pre-commit halt; align Build dispatch
extend REVIEWER_SYSTEM_PROMPT with two prepended HARD CONSTRAINTS:
- first action MUST be `git diff origin/<base>` (single-rev, captures
uncommitted). no other diff first; no checkout_pr; no alt-ref fetches;
no branch listing; no `gh pr list`.
- empty canonical diff + claimed-changes dispatch ⇒ reply exactly with
`no changes detected — likely pre-commit Build self-review;
orchestrator should commit then re-dispatch` and stop. do not guess
PR numbers (the zed thrash that ended in `checkout_pr({2582})`).
reshape Build mode reviewfrog dispatch step around a verbatim template
that names: (a) the situation is pre-commit, (b) canonical diff command,
(c) halt-on-empty-diff rule. orchestrator side now says the same thing
as the reviewer's baked-in prompt. delegation-discipline bullets and
orchestrator-evaluation guidance kept intact.
* checkout_pr: handle detached-HEAD entry in initial-branch invariant
pullfrog incremental review caught a defense-in-depth gap: `git
rev-parse --abbrev-ref HEAD` returns the sentinel string `"HEAD"` on
detached entry, which is the default `actions/checkout` state for
`pull_request` events. with the previous string-typed `initialBranch`,
both the captured value and the live probe would equal `"HEAD"` on
any detached state, trivially satisfying the invariant — including a
subagent doing `git checkout --detach <sha>`.
discriminate the captured HEAD: probe `git symbolic-ref --short HEAD`
first (works on named branches), fall back to `git rev-parse HEAD`
(SHA) on detached entry. store as
`{ kind: "branch"; name } | { kind: "detached"; sha }`. checkout_pr
runs the identical probe at call time and compares like-with-like
(branch name vs branch name, SHA vs SHA).
refusal error renders both heads via a small `describeHead` helper and
chooses the right `git checkout` recovery target (branch name or SHA).
no inline-discriminant `as` casts — uses a top-level `headsEqual` that
narrows via the discriminator.
* fix: 9 unaddressed log-audit / run-audit findings
Co-authored-by: Cursor <cursoragent@cursor.com>
#815 entryPost stdlib-only imports; #823 MCP timeoutMs on checkout_pr/shell;
#816 FREE_FALLBACK → opencode/big-pickle; #822 chunk GraphQL nodes ≤100;
#817/#821 vip_audit 404 skip paths; #813 longer serializable retries;
#818 run-context handler-entered log; #805 audit severity template.
* fix: update footer test for big-pickle fallback slug
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: anneal round 1 — ghaCore getState casing, post-hook timeout
Match @actions/core STATE_ key semantics (no uppercasing), cap
postApiFetch at 30s, trim serializable retries to stay under GitHub's
10s webhook window, log Clerk failures in getUserTokenByGithubLogin.
Co-authored-by: Cursor <cursoragent@cursor.com>
* revert: drop run-context handler log (#818 deferred)
The #692 client-side fix is already on main; residual SyntaxError hits
are old action pins. Per-request log added noise without fixing anything.
Co-authored-by: Cursor <cursoragent@cursor.com>
* document per-issue Closes syntax for audit PRs
GitHub only auto-closes the first issue when numbers are comma-separated;
/audits and AGENTS.md now require Closes before each issue number.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: anneal round 2 — outreach privacy, alert resilience, vertex cleanup
Filter private repos from VIP authority output, harden console alert
lines against DB failures, drop spoofable changesets body check, and
unset GOOGLE_APPLICATION_CREDENTIALS after vertex credential cleanup.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: drop codexHome re-export of detectCodexRefresh
Import detectCodexRefresh directly from codexRefreshDetect.ts everywhere;
rename the unit test file to match. codexHome.ts stays install-only.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: drop deprecated minimax-m2.5-free; add paid minimax-m2.5
Remove the deprecated free MiniMax promo from the catalog, docs, and
tests. BYOK fallback and picker copy stay on opencode/big-pickle. Add
opencode/minimax-m2.5 and openrouter/minimax-m2.5 for Zen BYOK and
Router. Pin #816 regressions with freeFallbackCatalog and runErrorRenderer
unit tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: hidden minimax-m2.5-free fallback for stored slugs
Re-add opencode/minimax-m2.5-free as a hidden fallback alias to big-pickle
so repos with the legacy slug still resolve as free. Drop live Zen API
experiment tests in freeFallbackCatalog.test.ts.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Derive the platform default from moonshotai/kimi-k2 openRouterResolve, add models.dev drift coverage, and promote repos on default-branch workflow pushes when still needs_setup.
* fix: prevent cross-PR push from subagent-induced branch switch
A workflow_dispatch run for zed-industries/cloud (workflow run 26036155393)
force-pushed the orchestrator's work onto an unrelated engineer's PR branch
(origin/reactivate-pro-plan, PR #2582). The orchestrator's reviewfrog subagent
called checkout_pr({pull_number: 2582}), which (1) moved the shared working
tree to pr-2582 and (2) persisted pushDest pointing at reactivate-pro-plan.
The orchestrator's subsequent commit + push_branch then clobbered the victim
PR. Recovery + disclosure in PR #2584.
Three compounding bugs closed here:
1. checkout_pr dirty-tree guard had a first-call hole: the previous condition
required ctx.toolState.issueNumber to already be set, so on workflow_dispatch
runs the first checkout_pr (commonly from a subagent) bypassed the guard
entirely. Now any PR switch with a dirty tree is refused, including the
first switch of a run. Idempotent same-PR re-checkouts are still absorbed
by alreadyOnBranch inside checkoutPrBranch.
2. push_branch trusted sticky pushDest blindly. Added a backstop: refuse
pushes where the local branch matches /^pr-(\d+)$/ AND pushDest.remoteBranch
differs from it AND the current run is not scoped to PR N (event.is_pr === true
&& event.issue_number === N). This catches subagent-induced silent branch
switches even if a future bug reintroduces a first-call hole in fix#1.
3. Build-mode self-review prompt told the orchestrator to ship "the output of
git diff" to the reviewer. The model in this run synthesized
git diff main...HEAD, which excludes uncommitted work — and Build self-review
runs BEFORE the commit, so the reviewer saw an empty diff and thrashed,
eventually calling checkout_pr on a random PR to find something to look at.
Prompt now specifies git diff origin/<base-branch> (two-dot, no HEAD),
which compares the working tree against the remote base.
Refs:
zed-industries/cloud workflow run 26036155393
zed-industries/cloud#2582 (victim)
zed-industries/cloud#2584 (disclosure)
* review: key dirty-tree guard on current branch + drop 'two-dot' misnomer
Address review feedback on PR #796.
1. checkout_pr dirty-tree guard now keys off the live current branch
(git rev-parse --abbrev-ref HEAD), not ctx.toolState.issueNumber.
issueNumber is ALSO set by get_issue / get_issue_comments /
get_issue_events, so a subagent doing get_issue(N) followed by
checkout_pr(N) on a dirty tree would have bypassed the original guard
(issueNumber === pull_number). The current branch is the actual
primitive for "would this call move HEAD" — querying it directly avoids
correlating on toolState that other tools write to.
2. modes.ts: drop the wrong "two-dot" label on git diff origin/<base>.
That's the single-rev form, not two-dot. Copilot was right that the
label was confusing/contradictory with the actually-shown command.
* opencode v2: harness adapted to opencode-ai 1.15+ SDK-v2 / Effect-ts CLI rewrite
Bumps `opencode-ai` from `1.1.56` → `1.15.1` and ports the harness to the
v2 NDJSON event contract. The legacy `opencode.ts` is kept as reference;
`opencode_v2.ts` is the active runner via `agents/index.ts`.
Why: `1.1.56` doesn't echo Gemini `thought_signature` back through the
MCP tool-call serializer, so direct-Google reviews 400 on the 3rd-ish
tool call. The fix only exists in the `1.14.x`+ line, which also ships
the SDK-v2 / Effect-ts CLI rewrite — taking the rewrite is mandatory.
Also unblocks the Codex ChatGPT-subscription auth path.
Surface area:
- drop `init` / `message` / `result` / `tool_result` event types and
handlers (no longer emitted at v1.14+ per upstream
`cli/cmd/run.ts:588-601`).
- `tool_use` is now a single event covering both `state.status:
"completed"` and `"error"`. duration / subagent-finish bookkeeping
moves from the v1 `tool_result` handler into the consolidated
`tool_use` handler.
- new `reasoning` event handler — gated on `--thinking`, surfaces
Gemini-3 / OpenAI / Anthropic thinking blocks. `--thinking` added to
`baseArgs`.
- drop `pendingTaskDispatches` FIFO + `knownNonTaskCallIDs` set: at
v1.15 the `task` tool callID is stable across the whole
`tool-input-* → tool-call → tool-result/tool-error` chain
(`session/processor.ts:282-330`). exact-match map is sufficient.
- drop `experimental.batch_tool: true` from injected config — declared
but inert at v1.15. re-add once upstream wires it back.
- bin path: `bin/opencode` → `bin/opencode.exe` (postinstall renames
the platform-specific binary into `opencode.exe` for every OS now).
Validated locally:
- `pnpm test` 610/610 ✓
- `pnpm play --raw` end-to-end with Anthropic via OpenRouter ✓
- `pnpm play --raw` with `google/gemini-3.1-pro-preview`: 6 tool calls,
multiple reasoning blocks visible, `set_output` propagates, exit 0 ✓
(this is the headline `thought_signature` fix)
- runtest opencode: smoke ✓, restricted ✓, nobash ✓, token-exfil ✓
- runtest opencode: skill-invoke and mcpmerge fail (model-behavior
drift on the new system prompt; wiring confirmed intact via direct
repro showing both `robinMCP` and `pullfrog` MCP tools exposed).
Tracked for follow-up; does not gate the migration.
Plugin (`opencodePlugin.ts`) and skill discovery paths are unchanged at
v1.15 — verified upstream and reused as-is. Bus subscription via
`bus.subscribeAll()` and the `event` hook still fan out every payload.
* model-smoke: bump opencode bin path to opencode.exe (v1.14+ rename)
The v1.14+ postinstall.mjs renames the platform-specific binary to
`bin/opencode.exe` for every OS (incl. linux/darwin), not just Windows.
Mirrors the fix in action/agents/opencode_v2.ts.
* opencode v2: set PWD env explicitly to fix skill / project-config discovery
Root cause for skill-invoke + mcpmerge harness regressions: opencode-ai 1.15
reads `process.env.PWD` first (with `process.cwd()` as fallback) when
resolving the SDK client's `directory` parameter — see upstream
`cli/cmd/run.ts:282`:
const root = Filesystem.resolve(process.env.PWD ?? process.cwd())
We pass `cwd: repoDir` to spawn, but the child inherits the harness's PWD
via `...process.env`. Under `pnpm runtest` (and `pnpm play`) PWD is the
`action/` directory, not the cloned test repo. Result: opencode creates
two instances per session — one at `process.cwd()` (correct) and one at
`PWD` (wrong) — and the agent's session runs in the PWD-derived one,
which can't see the project's `.opencode/skills/` or `.claude/skills/`.
Empirically traced via the full opencode stderr trace under the runtest
harness: `service=skill count=3 init` (no `pullfrog-skill-check`) plus a
second `service=default directory=<harness-pwd> creating instance` line
per run. With `PWD=repoDir` set explicitly, `count=4 init` includes the
test skill, the agent reaches for `skill({"name":"pullfrog-skill-check"})`
exactly as the validator expects, and mcpmerge's `robinMCP_get_test_value`
becomes accessible too.
Validated locally: skill-invoke-opencode ✓, mcpmerge-opencode ✓, smoke ✓,
restricted ✓, nobash ✓, token-exfil ✓ (flaked once on a model-narration
match, passes on retry; unrelated to PWD).
* opencode v2: drop ThinkingTimer; use opencode's reasoning.part.time directly
opencode-ai 1.15 emits `reasoning` parts with `time.start` / `time.end`
on terminal state (`cli/cmd/run.ts:671`), giving us a precise per-block
"thought for X s" duration straight from the runtime. The v1
ThinkingTimer heuristic — measuring wall-clock between markToolResult
and the next markToolCall — was an approximation when no native source
existed; with v2 it's redundant and noisy (it would log alongside the
real reasoning event, and conflated network latency with model thinking).
Removed: `ThinkingTimer` import, `thinkingTimers` Map, `timerFor()`
helper, both `markToolCall` / `markToolResult` call sites in `tool_use`.
The `reasoning` handler now reads `part.time.start/end` directly and
prefixes the visible preview with `(X.Ys)`.
Output before: `» thinking: <preview>` + `» thought for 4.0s` (separate)
Output now: `» thinking (4.0s): <preview>` (one line, sourced)
For models that don't emit reasoning (Sonnet without extended thinking,
GPT-4o, etc.), there's just no thinking line — which matches reality
better than the gap-heuristic, which would fire on any pause >3s
including provider-side latency that wasn't actual model reasoning.
Validated locally: skill-invoke ✓, mcpmerge ✓, smoke ✓, Gemini play
shows `» thinking (4.0s)` and `» thinking (0.8s)` from real durations.
* claude.ts: same PWD fix as opencode v2; entryPost: refresh stale comment
claude-code 2.1.x reads `process.env.PWD` and registers it as a "session"
additional-working-directory when it differs from `process.cwd()` (per the
bundled cli.js: `let H = process.env.PWD; if (H && H !== Y7() && ...)
j.set(H, { path: H, source: "session" })`). Without overriding PWD on the
spawn env, claude inherits the harness's PWD via `...process.env` — under
`pnpm runtest` / `pnpm play` that's `action/`, not the cloned test repo —
and adds the wrong dir to the agent's allowed working set.
Symmetric to the opencode v2 fix in 52337f9. Pre-empts the same class of
"agent's session sees the wrong cwd" failures on the claude side.
Also refresh the stale `action/agents/opencode.ts` reference in
entryPost.ts to point at opencode_v2.ts (the active runner), with the v1
file noted as kept-for-reference.
* opencode: extract shared helpers into opencodeShared.ts; v2 cleanup
Code-quality pass on the v2 work:
1. New `agents/opencodeShared.ts` (144 lines) for genuinely-shared helpers
between v1 and v2:
- `OpenCodeConfig` type
- `geminiHighThinkingOverrides()` (registry-driven Gemini thinking pin)
- `buildReviewerAgentConfig()` (reviewfrog config builder, was in v1
and re-imported by v2 via a back-reference)
- `installOpencodeCli({ binPath })` (parameterized — v1 passes
`bin/opencode`, v2 passes `bin/opencode.exe` via a per-version
`installCli` lambda; matches each pinned version's npm shape)
- `autoSelectModel()` + `getOpenCodeModels()` model-registry fallback
v2 drops the `import { ... } from "./opencode.ts"` back-reference; v1
keeps a one-line `export { geminiHighThinkingOverrides }` re-export
so `opencode.test.ts` keeps working unchanged. Once v1 is retired
(post burn-in) opencodeShared collapses back into v2.
2. `opencode_v2.ts` cleanup:
- drop dead state (`currentStepId`, `stepHistory` were write-only —
their reader was the v1 `tool_result` handler we deleted)
- hoist `state` in `tool_use` handler; replace nested-ternary payload
extraction with a `terminalPayload(state)` helper
- extract `formatPartDuration(time)` for the reasoning-block
"(X.Ys)" suffix
- tighten `OpenCodeBusEnvelopeEvent` type to include `tool` /
`callID` fields directly, drop the `partWithToolFields` cast
- trim docblocks per AGENTS.md "≤ 2-3 lines per code line": reasoning
handler, tool_use handler, bus envelope handler all shortened
- `step_start` becomes an explicit `() => {}` no-op so the dispatcher
doesn't log "unhandled event" for every step
3. `subagentRegistration.test.ts` retargeted at the new file split —
reads opencodeShared.ts for the buildReviewerAgentConfig assertions
and opencode_v2.ts for the orchestrator-model wire-through.
Net: -306 source lines (1339+1130 → 1228+1031+144). Tests + lint + format
+ typecheck all green; skill-invoke-opencode ✓ and smoke ✓ verified
against the refactored v2 runtime.
* opencode v2: address PR review feedback
Three fixes from the inline review threads on #767:
1. Activity-diagnostic ordering bug (Copilot review at L705): the chunk-
level `markActivity()` resets the module-level idle counter, so the
per-event `getIdleMs()` sample inside the dispatch loop was always
~0ms — the "no activity for Xs" diagnostic never fired. Replaced with
a runner-local `lastEventAt` so we measure real event-to-event silence
instead of chunk-arrival latency. Drop the unused `getIdleMs` import.
2. TDZ-defensive hoist (Pullfrog review nit): `agentErrorEvent`,
`lastProviderError`, and `recentStderr` are closed over by the
`handlers` const but were declared after it. No current bug because
handlers only fire inside the awaited `spawn()`, but a future
refactor that triggers a handler synchronously during setup would
surface a TDZ. Hoisted above `handlers`.
3. `step_finish.part.tokens.reasoning` follow-up (Pullfrog review at
L566): leave a `TODO` comment marking the gap until `AgentUsage`
grows a `reasoningTokens` field — separate PR with schema work.
Cost totals stay correct because `part.cost` is summed independently.
Other thread states for the record:
- Copilot L63 (geminiHighThinkingOverrides import from legacy): already
fixed by the opencodeShared.ts extraction in 83a7cab.
- Copilot L672 (ThinkingTimer over-reports on terminal events): already
fixed by dropping ThinkingTimer in a1e536b — we use opencode's own
`reasoning.part.time.{start,end}` for thinking durations now.
- Pullfrog L642 (onToolUse double-fire on subagent dispatch): re-checked
the bus-envelope flow; the plugin filters orchestrator events except
for status=running task dispatches, and bus-envelope returns before
calling handlers.tool_use on those. No double-fire under current code.
Validated: 610/610 unit tests, lint + format + typecheck clean,
skill-invoke-opencode ✓.
* DX: flip pnpm play / pnpm runtest to docker-by-default
Restores the script shape wiki/docker.md has documented since the docker
rewrite (#750). PR #756 inadvertently reverted action/package.json's
gha/play/runtest scripts to host-only and dropped the :local variants;
the wiki kept the new shape, so docs and reality drifted. The OpenCode-v2
migration agent ran `pnpm play --raw …` host-side throughout because the
host entry was the only thing that existed.
scripts (root → action):
- pnpm play → pnpm -C action gha play.ts (docker, default)
- pnpm play:local → pnpm -C action play:local (host)
- pnpm runtest → pnpm -C action gha test/run.ts (docker, default)
- pnpm runtest:local → pnpm -C action runtest:local (host)
- pnpm gha is restored in action/package.json (re-adds `node gha.ts`)
action/package.json deliberately ships only the :local variants — bare
`pnpm -C action play` now errors instead of silently bypassing docker.
This is a tradeoff per the user prompt's "consider whether NAMES should
change" hint: the explicit error is worth the small CI churn.
CI workflows: `.github/workflows/test.yml` and
`action/.github/workflows/test.yml` flipped from `pnpm runtest …` to
`pnpm runtest:local …`. Semantics unchanged — they still execute
`node test/run.ts` directly on the GHA Linux runner; nesting docker on
GHA is unnecessary overhead. Only the script name changed to match the
new package.json.
Webhook tester: the existing root `pnpm play` was actually a webhook
handler smoke harness (root play.ts), unrelated to the action runtime.
Renamed root play.ts → webhook.ts and exposed it as `pnpm webhook` so
`pnpm play` can carry the docker-by-default action shortcut without
collision. README updated.
File headers updated:
- action/play.ts: invocation block now points at `pnpm play` /
`pnpm play:local`
- action/test/run.ts: same
- action/gha.ts: usage block calls out the new shortcut wrappers
AGENTS.md: extended the existing "local sanity checks of action tool
logic" rule with the play / play:local / runtest / runtest:local
selection guidance and the `cd action; pnpm play` footgun note.
wiki/docker.md unchanged — already described the now-real shape.
* test/crossagent: add codex-auth smoke
Pins openai/gpt-5.5 (in opencode's Codex ALLOWED_MODELS) and runs the
full opencode harness against the env-provided CODEX_AUTH_JSON. Verifies:
- installCodexAuth() materializes auth.json under the test HOME
- opencode routes openai requests through ChatGPT subscription auth
(no OPENAI_API_KEY in env, AT path forced via expires: 0)
- the refresh chain advances during the run (refresh_token rotates)
- detectCodexRefresh() would surface the rotation to entryPost.ts
The post-hook write-back fetch isn't reachable from `pnpm runtest`
(it's a separate GHA `post:` step). The integration boundary that
matters end-to-end is "did the on-disk auth.json change in a way
detectCodexRefresh recognizes" — that's exactly what this test asserts.
CI wiring (already committed in a1c1fd4f as part of the DX flip):
- .github/workflows/test.yml: CODEX_AUTH_JSON via secrets in
action-agents env block
- action/.github/workflows/test.yml: same; codex-auth in the
hardcoded test matrix with a claude exclude
The provisioning step on the user's side is `gh secret set
CODEX_AUTH_JSON --repo pullfrog/app < auth.json`.
ci.test.ts: expectedAgentEnvVars now includes provider
`managedCredentials` so the "env vars cover all provider API keys"
invariant stays self-correcting as more managed credentials land.
* docs(codex-auth): make storage requirement unmissable
A previous reviewing agent on this branch came away thinking
`CODEX_AUTH_JSON` could live in GitHub Actions secrets. It can't —
`entryPost.ts` rewrites the rotated refresh token after every run, and GH
Actions secrets are immutable at runtime, so any non-Pullfrog-Postgres
storage breaks the refresh chain on the first rotation (~1h silent
expiry).
- wiki/codex-auth.md: prominent `[!IMPORTANT]` callout above the fold,
with the words "GitHub Actions secrets DO NOT WORK" verbatim and an
enumeration of broken alternatives.
- action/utils/codexHome.ts + action/entryPost.ts: header comments now
loudly contrast Pullfrog secret store vs GH Actions and explain the
writeback constraint.
- AGENTS.md: terse one-bullet rule next to the model-resolution rule so
future agents don't repeat the mistake.
- .github/workflows/test.yml + action/.github/workflows/test.yml: added a
comment marking the existing `secrets.CODEX_AUTH_JSON` injection as a
CI smoke-testing shortcut, not the canonical pattern. CI wiring itself
unchanged per scope.
* auth codex: auto-open device URL, drop --scope flag
- detect `https://auth.openai.com/codex/device...` from codex CLI output
and best-effort launch it in the user's default browser (open / xdg-open
/ cmd start, wslview fallback on linux). gated so we only open once per
flow; failures are swallowed so manual copy-paste still works.
- drop the `--scope` flag entirely. the device-code flow is fundamentally
interactive (browser approval), so a "skip-the-prompt" flag for just one
of the prompts was dead weight. collapses scope selection to "always
prompt on org-owned, always account on user-owned".
* rename gha→docker, flip play/runtest defaults to host
the previous shape conflated "real GitHub Actions" with the local docker
container that mocks it, and made the slow docker path the default for
fast-iteration scripts.
- `action/gha.ts` → `action/docker.ts` (banner, --doctor, --help, image
tag `pullfrog-docker:*`, volume `pullfrog-docker-node-modules-*`,
tmpdir, error messages)
- `pnpm play` / `pnpm runtest` now default to host (fast iteration);
`pnpm play:docker` / `pnpm runtest:docker` run inside the container
- `pnpm gha` → `pnpm docker` (the container runner shortcut)
- `pnpm webhook` → `pnpm play:webhook` (fits the play: namespace; the
bare name implied a webhook server, which hookdeck-cli already is)
- update docs (`wiki/{docker,action-tests,billing,adversarial,browser}.md`,
`README.md`, `AGENTS.md`), CI workflows
(`.github/workflows/test.yml`, `action/.github/workflows/test.yml`),
and code headers (`action/{play,test/run,utils/runFixture}.ts`,
`webhook.ts`, `action/test/coverage.ts`)
`action/commands/gha.ts` keeps its name — it's the real GitHub Actions
entry point for the `pullfrog gha` CLI command (not the docker mock).
* fix(codex): route post-hook writeback through apiFetch + conditional skip
Three threads addressing PR #767 followups.
action/entryPost.ts: replace raw fetch() with apiFetch() so the
PUT /api/runtime/secret call carries the x-vercel-protection-bypass
header/query when targeting a preview deployment. raw fetch silently
401s against the Vercel SSO gate, so every preview-env Codex run was
losing its rotated refresh token. production is unaffected (no SSO).
action/test/crossagent/codexAuth.ts: gate the test on CODEX_AUTH_JSON
via new TestRunnerOptions.skipIf hook. when the secret is absent
(forks, contributors without it), runTestForAgent short-circuits to a
passing-with-skipped ValidationResult before any agent spawn — so the
matrix's fail-fast: true setting doesn't cascade-cancel siblings. CI
on pullfrog/app and dev-local with .env both still run the test for
real. printSingleValidation/printResults now render skipped entries
distinctly.
doc/comment drift:
- docs/codex-auth.mdx, wiki/codex-auth.md: drop stale --scope flag
mention (removed in 10be96db, scope is now always interactively
prompted or implicit).
- wiki/codex-auth.md: tighten Claude-defense wording — materialization
is agent-gated (opencode/opencode_v2 harness), not model-gated;
opencode runs with non-OpenAI models still materialize the file,
it's just not read.
- action/Dockerfile, action/docker-entrypoint.sh: pnpm gha / gha.ts
→ pnpm docker / docker.ts (renamed in a2a63929).
- app/api/runtime/secret/route.ts: refer to the save-time scope prompt
instead of the dropped --scope flag.
* smoke: force ≥2 tool calls; document test-bar in wiki + AGENTS
upgrade crossagent/smoke prompt to call pullfrog_git status before
set_output. this exercises the 2nd model→agent round-trip across every
providers-live flagship, catching bugs like the Gemini thought_signature
echo that single-tool-call tests can't see.
also adds the "bar for adding new LLM-driven tests" section to
wiki/action-tests.md and an extension to the existing AGENTS.md
no-tests rule pointing at it — prefer upgrading existing matrix entries
over adding new ones.
local: pnpm runtest smoke opencode passes against both
anthropic/claude-sonnet-4-6 and google/gemini-pro.
---------
Co-authored-by: Colin McDonnell <colinmcd94@M1chelle.local>
wolfy
Colin McDonnell