5caeb75344
* review: 0-or-2+ lens rule, parallel-or-bust, downshifted subagent models
PR review wall-time was dominated by two failure modes: orchestrator
serial-dispatching subagents (despite prompt asking for parallel) and
running every lens on the same Opus tier as the orchestrator. Sample of
recent runs showed 25-60min reviews on small PRs, with 8-10min idle
gaps between subagent dispatches.
Three changes:
1. `action/modes.ts` — replace the soft "1 trivial / 2-3 typical /
4-5 high-stakes" lens calibration with a binary 0-or-2+ rule. Default
is 0 lenses (orchestrator handles review solo with optional cheap
tracerfrog dispatches). 2+ parallel lenses only fire for substantive
PRs (>5 files AND >200 lines) or high-stakes-subsystem touches. Never
exactly one. Both Review and IncrementalReview prompts get loud
ALL-CAPS framing on parallel dispatch — emit ALL Task tool_use blocks
in a single assistant turn before reading any result. Drop the
"do NOT lens-review the diff yourself" advice; orchestrator pulls
context aggressively, in parallel with the lens fan-out.
2. New `tracerfrog` subagent for mechanical code tracing ("where is X
used / who calls Y / what depends on Z"). Pure read+grep+report with
no judgment — orchestrator can dispatch many tracers cheaply in
parallel. Defined in `action/agents/reviewer.ts`. Wired into both
claude.ts (`--agents` JSON) and opencode.ts (`agent` config block).
3. Per-subagent model downshifts via `deriveSubagentModels`:
- Anthropic: reviewfrog → Sonnet, tracerfrog → Haiku
- OpenAI: both → gpt-5.4-mini
- other providers (xai, deepseek, gemini, etc.): inherit (no
standard tier triplet to downshift to)
Claude Code path always runs Anthropic so the downshift is hardcoded
inline in claude.ts. OpenCode uses the helper since orchestrator
provider varies.
Both runtimes' subagent-definition formats verified directly against
their source: `--agents` JSON `model` field (claude-code's
`AgentJsonSchema` accepts model+effort+maxTurns+more) and OpenCode's
`agent.{name}.model` config field (parsed via Provider.parseModel,
applied per-task in tool/task.ts line 92). Parallel dispatch is
infra-supported in both — only the orchestrator model's tool_use
emission pattern was the bottleneck.
Tests: subagentModels.test.ts (14 tests covering provider matrix),
subagentRegistration.test.ts (6 source-asserts catching shape
regressions in buildAgentsJson / buildReviewerAgentConfig).
* subagentModels: add openrouter routes (proxy/router mode)
Initial helper missed the openrouter prefix used by Pullfrog's router
proxy. preview-710 e2e showed the OpenCode + openrouter path receiving
no downshift — orchestrator and lenses both ran on opus-4.7 because
'openrouter/anthropic/claude-opus-4.7' didn't match any of the
anthropic/openai prefixes the helper checked.
Add explicit branches for 'openrouter/anthropic/...' (uses dot notation:
claude-sonnet-4.6 / claude-haiku-4.5) and 'openrouter/openai/...'
(gpt-5.4-mini for both reviewer and tracer). Same opus->sonnet,
sonnet->keep-but-haiku-tracer, haiku->no-op semantics as the direct
anthropic path.
* opencode: log resolved subagent models at startup
So we can verify per-subagent model overrides actually take effect at
runtime. Prints once per run alongside the existing model/effort log
lines.
* drop tracerfrog: keep reviewfrog only, LSP-powered tracer planned later
Removes the cheap-haiku-tracer subagent (TRACER_AGENT_NAME +
TRACER_SYSTEM_PROMPT, registrations in claude.ts/opencode.ts, dispatch
guidance in modes.ts). The mechanical-tracing use case will be served
better by an LSP-powered tool than by a separately-prompted subagent.
deriveSubagentModels collapses to a single { reviewer } shape; the
reviewfrog-on-Sonnet downshift stays. Same source-assert + provider-
matrix tests, minus the tracer-specific cases.
modes.ts wording: drop the 'subagent type cheat sheet' bullet, drop
the parenthetical 'often better served by tracerfrog than reviewfrog'
on the impact lens, drop tracerfrog from the same-turn-context-pulling
hint. The 0-or-2+ rule and ALL-CAPS parallel emphasis are unchanged.
* subagentModels: broader downshift coverage (gpt-pro, gemini-pro, grok); drop gpt-mini target
Scanned every resolved orchestrator slug in action/models.ts against
models.dev pricing data. Identified five clear cases where the
orchestrator is meaningfully expensive AND has a cheaper sibling that
remains capable enough for review-style judgment work.
Changes:
- Anthropic: opus → sonnet (kept; -40%)
- OpenAI: gpt → gpt-5.4 (was: gpt-mini; -54% instead of -85% but
preserves review-quality judgment — gpt-mini was too dumb)
- OpenAI: gpt-pro → gpt (NEW; -93%, biggest single unlock —
gpt-5.5-pro is $30/Mtok in vs gpt-5.5 at $5)
- Google: gemini-pro → gemini-flash (NEW; -75%)
- xAI: grok-4.3 → grok-4-1-fast (NEW; -80%)
Every branch handles the three routes in use: direct provider slug,
opencode-vendored, and openrouter-proxied. Variants below the downshift
target (mini/nano/flash/fast/sonnet/haiku) inherit (no further drop).
Skipped:
- DeepSeek: v4-flash ($0.14/Mtok) is too far below review judgment
threshold; v4-pro orchestrator already cheap ($0.55 blended).
- Moonshot: kimi-k2-thinking would only save 32% and slug stability on
OpenRouter is uncertain; revisit if cost matters.
- o3: already mid-tier in OpenAI's reasoning family; no clean target.
* models: hoist subagent downshift into the registry, add hidden flag
The downshift relationship now lives next to each alias's resolve /
openRouterResolve as a sibling field. Two new ModelDef fields:
- subagentModel?: string — alias key (within same provider) of the
cheaper sibling reviewfrog should use as a lens-fanout subagent.
e.g. claude-opus → 'claude-sonnet'.
- hidden?: boolean — exclude from selectable lists (UI dropdown,
CLI init picker). Does NOT affect resolution; for that use
fallback. Used so internal-only subagent targets like openai/gpt-5.4
exist in the registry but never appear as a user-facing pick.
Wiring:
- anthropic.claude-opus → claude-sonnet (-40%)
- openai.gpt-pro → gpt (-93%, biggest unlock)
- openai.gpt → gpt-5.4 (-54%); gpt-5.4 added with hidden:true
- google.gemini-pro → gemini-flash (-75%)
- mirrored across opencode + openrouter providers (each provider
declares its own three-route data so the downshift declaration
is colocated with the rest of the alias definition).
deriveSubagentModels collapses from ~85 lines of prefix-matching to
a ~15-line registry reverse-lookup: find the alias whose resolve OR
openRouterResolve matches the orchestrator's spec, follow its
subagentModel pointer, return the matching field of the target alias.
Filter sites updated:
- components/ModelSelector.tsx: !a.fallback && !a.hidden
- action/commands/init.ts: same
Tests rewritten to exercise the registry through the public surface;
the matrix collapses to one assertion per (provider × route) pair.
* TEMP: log per-step cost+tokens for subagent model verification (PR #710)
* TEMP: also log SUBAGENT step_finish from bus envelope handler
* remove temporary per-step diagnostic logs (verification done)
Verified subagent model downshift takes effect end-to-end on the OpenCode
+ openrouter path. PR #8 in pullfrog/preview-710-review-perf dispatched
3 lenses (billing-subsystem / security / correctness) on the orchestrator's
opus-4.7 session, and per-subagent step_finish events showed actual cost
exactly matching Sonnet pricing rates (60% of what Opus would have cost):
session n actual if-Opus if-Sonnet match
T3VrUuF... 5 $0.2425 $0.4042 $0.2425 Sonnet ✓
93ZZR7E... 4 $0.2253 $0.3754 $0.2253 Sonnet ✓
Fb1Kr7b... 4 $0.2495 $0.4158 $0.2495 Sonnet ✓
The startup '» subagent models: reviewfrog=...' line stays — useful
permanent diagnostic showing the resolved subagent model per-run.
* TEMP: log per-event model from claude.ts assistant handler
* remove temporary per-event model log (claude.ts verification done)
Verified subagent model downshift takes effect end-to-end on the Claude
Code path. PR #9 in pullfrog/preview-710-review-perf dispatched 2 lenses
on an opus-4-7 orchestrator. Per-assistant-event model field from the
SDK's stream-json output, partitioned by parent_tool_use_id:
ORCH (parent_tool_use_id=null): 17 events all model=claude-opus-4-7
SUBAGENT lens:billing-subsystem: 17 events all model=claude-sonnet-4-6
SUBAGENT lens:security: 21 events all model=claude-sonnet-4-6
Zero leakage to opus from either subagent session. The per-subagent
'model' field in --agents JSON is honored by claude-code at the SDK
level, identical to the OpenCode path verified earlier.
* opencode: bump per-call output cap 5K → 16K to unblock large reviews
The 5K cap (added in #616 to lower OpenRouter upfront budget reservation
for low-wallet runs) was capping the entire response of a single LLM call,
not just the budget reservation. A single tool_use response — like a
`create_pull_request_review` with many inline comments — would truncate
mid-stream past 5K output tokens, leave the JSON unparseable, and the tool
would never actually invoke. We hit this on PR #710's verify-downshift PR:
review aggregated from 3 lenses had 11 inline comments + a long body,
truncated at out=5000 on every retry attempt, action exited with 'Review
mode finished without calling create_pull_request_review after 3 retry
attempts'.
Investigated whether OpenCode (or OpenAI/Anthropic/OpenRouter directly)
exposes a separate budget-reservation parameter that could stay small
while letting the response exceed it. They don't — `max_tokens` /
`max_completion_tokens` is the single value all four use for both the
upfront reservation and the hard output ceiling. No way to decouple them
at the API surface.
Bumped to 16K as a middle ground: 8× the prior cap (handles every review
shape we've observed plus headroom), still half of OpenCode's 32K default
so the wallet-burn benefit for low-balance accounts is preserved, just
smaller. For Opus 4.7 a typical ~50K-input call now reserves roughly
$0.65 instead of the prior $0.38.
Updated the constant comment to spell out the trade-off clearly so this
doesn't happen again.
* opencode: drop OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX override entirely
Verified the original rationale for the override is obsolete. From #616
the cap shrunk OpenRouter's per-call upfront budget reservation so a
single call's reservation wouldn't exceed the per-run key cap
(`ROUTER_PER_RUN_LIMIT_USD = 25`) and lock low-balance accounts out of
starting a run.
That per-run gate is gone. `app/api/proxy-token/route.ts` ~line 422
explicitly says: 'No upper cap (the old ROUTER_PER_RUN_LIMIT_USD = 25 is
gone). The natural ceiling is whatever the user has + their buffer.'
Router now mints keys with `keyLimitCents = balance + buffer` ($50 for
autoreload+card, $5 for card-only, $0 for no-card). A single call's
upfront reservation fits comfortably within that — no separate per-call
gate to fail past.
The cap had a real downside as a hard per-call output truncation. A
single `create_pull_request_review` tool_use with many inline comments
would truncate mid-stream past 5K output tokens, the JSON would be
unparseable, and the tool never invoked. Hit on PR #710's
verify-downshift PR.
Removing the override entirely; OpenCode falls back to its 32K default.
Left an explanatory note above the env-var assignment site so the next
person doesn't unknowingly re-add it.
969 lines
29 KiB
TypeScript
969 lines
29 KiB
TypeScript
import { execFileSync } from "node:child_process";
|
|
import * as p from "@clack/prompts";
|
|
import arg from "arg";
|
|
import pc from "picocolors";
|
|
import { modelAliases, type ProviderConfig, providers, resolveDisplayAlias } from "../models.ts";
|
|
|
|
const PULLFROG_API_URL = (process.env.PULLFROG_API_URL || "https://pullfrog.com").replace(
|
|
/\/+$/,
|
|
""
|
|
);
|
|
|
|
function link(text: string, url: string): string {
|
|
return `\x1b]8;;${url}\x07${text}\x1b]8;;\x07`;
|
|
}
|
|
|
|
type CliProvider = {
|
|
id: string;
|
|
name: string;
|
|
envVars: readonly string[];
|
|
models: { value: string; label: string; hint?: string | undefined }[];
|
|
};
|
|
|
|
function buildProviders(): CliProvider[] {
|
|
return Object.entries(providers)
|
|
.filter(([key]) => key !== "opencode" && key !== "openrouter")
|
|
.map(([key, config]: [string, ProviderConfig]) => {
|
|
const aliases = modelAliases.filter((a) => a.provider === key && !a.fallback && !a.hidden);
|
|
const recommended = aliases.find((a) => a.preferred);
|
|
const sorted = [...aliases].sort((a, b) => {
|
|
if (a.preferred && !b.preferred) return -1;
|
|
if (!a.preferred && b.preferred) return 1;
|
|
return 0;
|
|
});
|
|
return {
|
|
id: key,
|
|
name: config.displayName,
|
|
envVars: config.envVars,
|
|
models: sorted.map((a) => ({
|
|
value: a.slug,
|
|
label: a.displayName,
|
|
hint: a === recommended ? "recommended" : undefined,
|
|
})),
|
|
};
|
|
});
|
|
}
|
|
|
|
const CLI_PROVIDERS = buildProviders();
|
|
|
|
function resolveModelProvider(slug: string): CliProvider | null {
|
|
const providerId = slug.split("/")[0];
|
|
return CLI_PROVIDERS.find((p) => p.id === providerId) ?? null;
|
|
}
|
|
|
|
// ── helpers ──
|
|
|
|
// active spinner reference so bail/catch can clean up the terminal
|
|
let activeSpin: ReturnType<typeof p.spinner> | null = null;
|
|
|
|
function bail(msg: string): never {
|
|
if (activeSpin) {
|
|
activeSpin.stop(pc.red("failed"));
|
|
activeSpin = null;
|
|
}
|
|
p.cancel(msg);
|
|
process.exit(1);
|
|
}
|
|
|
|
function handleCancel<T>(value: T | symbol): asserts value is T {
|
|
if (p.isCancel(value)) {
|
|
if (activeSpin) {
|
|
activeSpin.stop(pc.red("canceled."));
|
|
activeSpin = null;
|
|
}
|
|
p.cancel("canceled.");
|
|
process.exit(0);
|
|
}
|
|
}
|
|
|
|
function getGhToken(): string {
|
|
let token: string;
|
|
try {
|
|
token = execFileSync("gh", ["auth", "token"], { encoding: "utf-8" }).trim();
|
|
} catch {
|
|
bail(
|
|
`gh cli not found or not authenticated.\n` +
|
|
` ${pc.dim("install:")} https://cli.github.com\n` +
|
|
` ${pc.dim("then:")} gh auth login`
|
|
);
|
|
}
|
|
if (!token) {
|
|
bail(
|
|
`gh cli returned an empty token. try re-authenticating:\n` +
|
|
` ${pc.dim("run:")} gh auth login`
|
|
);
|
|
}
|
|
return token;
|
|
}
|
|
|
|
type GhApiResult<T = unknown> = { data: T; scopes: string | null };
|
|
|
|
async function ghApi<T = unknown>(path: string, token: string): Promise<GhApiResult<T>> {
|
|
const controller = new AbortController();
|
|
const timeout = setTimeout(() => controller.abort(), 30_000);
|
|
try {
|
|
const response = await fetch(`https://api.github.com${path}`, {
|
|
headers: {
|
|
authorization: `Bearer ${token}`,
|
|
accept: "application/vnd.github+json",
|
|
"x-github-api-version": "2022-11-28",
|
|
},
|
|
signal: controller.signal,
|
|
});
|
|
|
|
if (!response.ok) {
|
|
const body = await response.text().catch(() => "");
|
|
throw new Error(`github api ${path} returned ${response.status}: ${body}`);
|
|
}
|
|
|
|
const data = (await response.json().catch(() => {
|
|
throw new Error(`github api ${path} returned non-JSON response`);
|
|
})) as T;
|
|
return { data, scopes: response.headers.get("x-oauth-scopes") };
|
|
} finally {
|
|
clearTimeout(timeout);
|
|
}
|
|
}
|
|
|
|
function parseGitRemote(): { owner: string; repo: string } {
|
|
let url: string;
|
|
try {
|
|
url = execFileSync("git", ["remote", "get-url", "origin"], { encoding: "utf-8" }).trim();
|
|
} catch {
|
|
bail("not a git repository or no 'origin' remote found.");
|
|
}
|
|
|
|
const match = url.match(/github\.com(?::\d+)?[:/]+([^/]+)\/(.+?)(?:\.git)?(?:\/)?$/);
|
|
if (!match) bail(`could not parse github owner/repo from remote: ${url}`);
|
|
return { owner: match[1], repo: match[2] };
|
|
}
|
|
|
|
function openBrowser(url: string) {
|
|
try {
|
|
const platform = process.platform;
|
|
if (platform === "darwin") execFileSync("open", [url], { stdio: "ignore" });
|
|
else if (platform === "win32")
|
|
execFileSync("cmd", ["/c", "start", "", url], { stdio: "ignore" });
|
|
else execFileSync("xdg-open", [url], { stdio: "ignore" });
|
|
} catch {
|
|
// headless/SSH — user will open the URL manually
|
|
}
|
|
}
|
|
|
|
// ── Pullfrog API ──
|
|
|
|
type SecretsApiData = {
|
|
error?: string;
|
|
appSlug?: string;
|
|
installationId?: number | null;
|
|
repositorySelection?: string | null;
|
|
isOrg?: boolean;
|
|
accessible?: boolean;
|
|
repoSecrets?: string[];
|
|
orgSecrets?: string[];
|
|
pullfrogSecrets?: string[];
|
|
repoStatus?: string | null;
|
|
repoModel?: string | null;
|
|
hasRuns?: boolean;
|
|
};
|
|
|
|
type SecretsInfo = {
|
|
isOrg: boolean;
|
|
installationId: number | null;
|
|
secretsAccessible: boolean;
|
|
repoSecrets: string[];
|
|
orgSecrets: string[];
|
|
pullfrogSecrets: string[];
|
|
model: string | null;
|
|
hasRuns: boolean;
|
|
};
|
|
|
|
type InstallationNotFound = {
|
|
appSlug: string;
|
|
installationId: number | null;
|
|
repositorySelection: "all" | "selected" | null;
|
|
isOrg: boolean;
|
|
};
|
|
|
|
type StatusResult =
|
|
| ({ installed: true } & SecretsInfo)
|
|
| ({ installed: false } & InstallationNotFound);
|
|
|
|
type SessionApiData = {
|
|
id?: string;
|
|
installed?: boolean;
|
|
error?: string;
|
|
};
|
|
|
|
type SetupApiData = {
|
|
error?: string;
|
|
success?: boolean;
|
|
already_existed?: boolean;
|
|
pull_request_url?: string;
|
|
commit_url?: string;
|
|
hash?: string;
|
|
};
|
|
|
|
type DispatchApiData = {
|
|
error?: string;
|
|
url?: string;
|
|
};
|
|
|
|
type ApiResult<T = Record<string, unknown>> = { ok: boolean; status: number; data: T };
|
|
|
|
async function pullfrogApi<T = Record<string, unknown>>(ctx: {
|
|
path: string;
|
|
token: string;
|
|
method?: string;
|
|
body?: Record<string, unknown>;
|
|
}): Promise<ApiResult<T>> {
|
|
const headers: Record<string, string> = { authorization: `Bearer ${ctx.token}` };
|
|
if (ctx.body) headers["content-type"] = "application/json";
|
|
const controller = new AbortController();
|
|
const timeout = setTimeout(() => controller.abort(), 30_000);
|
|
try {
|
|
const response = await fetch(`${PULLFROG_API_URL}${ctx.path}`, {
|
|
method: ctx.method || "GET",
|
|
headers,
|
|
body: ctx.body ? JSON.stringify(ctx.body) : null,
|
|
signal: controller.signal,
|
|
});
|
|
const data = (await response.json().catch(() => ({}))) as T;
|
|
return { ok: response.ok, status: response.status, data };
|
|
} finally {
|
|
clearTimeout(timeout);
|
|
}
|
|
}
|
|
|
|
async function fetchStatus(ctx: {
|
|
token: string;
|
|
owner: string;
|
|
repo: string;
|
|
}): Promise<StatusResult> {
|
|
const result = await pullfrogApi<SecretsApiData>({
|
|
path: `/api/cli/secrets?owner=${encodeURIComponent(ctx.owner)}&repo=${encodeURIComponent(ctx.repo)}`,
|
|
token: ctx.token,
|
|
});
|
|
|
|
if (!result.ok) {
|
|
const errorMsg = result.data.error || "";
|
|
if (result.status === 401) bail("invalid or expired github token.");
|
|
if (result.status === 404) {
|
|
const sel = result.data.repositorySelection;
|
|
if (!result.data.appSlug) bail("server did not return appSlug");
|
|
return {
|
|
installed: false,
|
|
appSlug: result.data.appSlug,
|
|
installationId:
|
|
typeof result.data.installationId === "number" ? result.data.installationId : null,
|
|
repositorySelection: sel === "all" || sel === "selected" ? sel : null,
|
|
isOrg: result.data.isOrg === true,
|
|
};
|
|
}
|
|
bail(errorMsg || `secrets check failed (${result.status})`);
|
|
}
|
|
|
|
return {
|
|
installed: true,
|
|
isOrg: result.data.isOrg === true,
|
|
installationId:
|
|
typeof result.data.installationId === "number" ? result.data.installationId : null,
|
|
secretsAccessible: result.data.accessible !== false,
|
|
repoSecrets: result.data.repoSecrets || [],
|
|
orgSecrets: result.data.orgSecrets || [],
|
|
pullfrogSecrets: result.data.pullfrogSecrets || [],
|
|
model: result.data.repoModel ?? null,
|
|
hasRuns: result.data.hasRuns === true,
|
|
};
|
|
}
|
|
|
|
// ── sessions ──
|
|
|
|
async function createSession(ctx: {
|
|
token: string;
|
|
owner: string;
|
|
repo: string;
|
|
}): Promise<string | null> {
|
|
try {
|
|
const result = await pullfrogApi<SessionApiData>({
|
|
path: "/api/cli/session",
|
|
token: ctx.token,
|
|
method: "POST",
|
|
body: { owner: ctx.owner.toLowerCase(), repo: ctx.repo.toLowerCase() },
|
|
});
|
|
if (!result.ok || !result.data.id) return null;
|
|
return result.data.id;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
type PollResult = "installed" | "pending" | "expired";
|
|
|
|
async function pollSession(ctx: { token: string; sessionId: string }): Promise<PollResult> {
|
|
const result = await pullfrogApi<SessionApiData>({
|
|
path: `/api/cli/session/${ctx.sessionId}`,
|
|
token: ctx.token,
|
|
});
|
|
if (result.status === 410) return "expired";
|
|
if (!result.ok) return "pending";
|
|
return result.data.installed === true ? "installed" : "pending";
|
|
}
|
|
|
|
function cleanupSession(ctx: { token: string; sessionId: string }) {
|
|
void pullfrogApi({
|
|
path: `/api/cli/session/${ctx.sessionId}`,
|
|
token: ctx.token,
|
|
method: "DELETE",
|
|
}).catch(() => {});
|
|
}
|
|
|
|
// ── installation ──
|
|
|
|
const SESSION_POLL_MS = 750;
|
|
const FALLBACK_POLL_MS = 5_000;
|
|
const HINT_AFTER_MS = 10_000;
|
|
const TIMEOUT_MS = 3 * 60 * 1000;
|
|
|
|
function listenForKey(key: string) {
|
|
let triggered = false;
|
|
const onData = (data: Buffer) => {
|
|
if (data.toString().toLowerCase() === key) triggered = true;
|
|
};
|
|
process.stdin.setRawMode?.(true);
|
|
process.stdin.resume();
|
|
process.stdin.on("data", onData);
|
|
return {
|
|
consume() {
|
|
if (!triggered) return false;
|
|
triggered = false;
|
|
return true;
|
|
},
|
|
stop() {
|
|
process.stdin.removeListener("data", onData);
|
|
process.stdin.setRawMode?.(false);
|
|
process.stdin.pause();
|
|
},
|
|
};
|
|
}
|
|
|
|
function installationConfigUrl(ctx: { owner: string; installationId: number; isOrg: boolean }) {
|
|
return ctx.isOrg
|
|
? `https://github.com/organizations/${ctx.owner}/settings/installations/${ctx.installationId}`
|
|
: `https://github.com/settings/installations/${ctx.installationId}`;
|
|
}
|
|
|
|
async function ensureInstallation(ctx: {
|
|
token: string;
|
|
owner: string;
|
|
repo: string;
|
|
}): Promise<SecretsInfo> {
|
|
activeSpin!.start("checking pullfrog app installation");
|
|
|
|
const initial = await fetchStatus(ctx);
|
|
if (initial.installed) {
|
|
activeSpin!.stop(`pullfrog app is installed on ${pc.cyan(`@${ctx.owner}`)}`);
|
|
if (initial.installationId) {
|
|
const configUrl = installationConfigUrl({
|
|
owner: ctx.owner,
|
|
installationId: initial.installationId,
|
|
isOrg: initial.isOrg,
|
|
});
|
|
process.stdout.write(`${pc.gray(p.S_BAR)} ${link(pc.dim(configUrl), configUrl)}\n`);
|
|
}
|
|
return initial;
|
|
}
|
|
|
|
const sessionId = await createSession(ctx);
|
|
|
|
if (initial.installationId) {
|
|
const repoRef = pc.bold(`${ctx.owner}/${ctx.repo}`);
|
|
const configUrl = installationConfigUrl({
|
|
owner: ctx.owner,
|
|
installationId: initial.installationId,
|
|
isOrg: initial.isOrg,
|
|
});
|
|
activeSpin!.stop(`pullfrog is installed on selected repos, but ${repoRef} is not included.`);
|
|
p.log.info(
|
|
`add it under "Repository access" on the installation config page.\n ${pc.dim(configUrl)}`
|
|
);
|
|
const openIt = await p.confirm({ message: "open browser?", active: "yes", inactive: "no" });
|
|
handleCancel(openIt);
|
|
if (openIt) openBrowser(configUrl);
|
|
} else {
|
|
activeSpin!.stop("pullfrog app not installed");
|
|
const installUrl = `https://github.com/apps/${initial.appSlug}/installations/select_target?state=cli`;
|
|
p.log.info(`opening browser to install...\n ${pc.dim(installUrl)}`);
|
|
openBrowser(installUrl);
|
|
}
|
|
|
|
const isRepoAccessUpdate = !!initial.installationId;
|
|
const baseMsg = isRepoAccessUpdate
|
|
? "once you've added the repo, onboarding will proceed automatically"
|
|
: "once you've installed the app, onboarding will proceed automatically";
|
|
activeSpin!.start(baseMsg);
|
|
|
|
let activeSessionId = sessionId;
|
|
let pollMs = activeSessionId ? SESSION_POLL_MS : FALLBACK_POLL_MS;
|
|
const listener = listenForKey("r");
|
|
const startedAt = Date.now();
|
|
let hintShown = false;
|
|
|
|
try {
|
|
while (Date.now() - startedAt < TIMEOUT_MS) {
|
|
await new Promise((r) => setTimeout(r, pollMs));
|
|
|
|
if (!hintShown && Date.now() - startedAt > HINT_AFTER_MS) {
|
|
activeSpin!.message(`${baseMsg} ${pc.dim("(press r to recheck manually)")}`);
|
|
hintShown = true;
|
|
}
|
|
|
|
const doneMsg = isRepoAccessUpdate ? "repo access confirmed" : "pullfrog app installed";
|
|
|
|
if (listener.consume()) {
|
|
activeSpin!.message("rechecking via GitHub API");
|
|
try {
|
|
const status = await fetchStatus(ctx);
|
|
if (status.installed) {
|
|
if (activeSessionId) cleanupSession({ token: ctx.token, sessionId: activeSessionId });
|
|
activeSpin!.stop(doneMsg);
|
|
return status;
|
|
}
|
|
} catch {
|
|
// network error — keep going
|
|
}
|
|
activeSpin!.message(`${baseMsg} ${pc.dim("(press r to recheck manually)")}`);
|
|
continue;
|
|
}
|
|
|
|
if (activeSessionId) {
|
|
// fast path: lightweight DB session poll (no GitHub API calls)
|
|
try {
|
|
const result = await pollSession({ token: ctx.token, sessionId: activeSessionId });
|
|
if (result === "expired") {
|
|
activeSessionId = null;
|
|
pollMs = FALLBACK_POLL_MS;
|
|
continue;
|
|
}
|
|
if (result === "installed") {
|
|
const status = await fetchStatus(ctx);
|
|
if (status.installed) {
|
|
cleanupSession({ token: ctx.token, sessionId: activeSessionId });
|
|
activeSpin!.stop(doneMsg);
|
|
return status;
|
|
}
|
|
}
|
|
} catch {
|
|
// transient error — keep polling
|
|
}
|
|
} else {
|
|
// no session available — poll fetchStatus directly at slower interval
|
|
try {
|
|
const status = await fetchStatus(ctx);
|
|
if (status.installed) {
|
|
activeSpin!.stop(doneMsg);
|
|
return status;
|
|
}
|
|
} catch {
|
|
// transient error — keep polling
|
|
}
|
|
}
|
|
}
|
|
} finally {
|
|
listener.stop();
|
|
}
|
|
|
|
if (activeSessionId) cleanupSession({ token: ctx.token, sessionId: activeSessionId });
|
|
bail(
|
|
isRepoAccessUpdate
|
|
? "timed out waiting for repo access.\n" +
|
|
` ${pc.dim("add the repo, then re-run:")} npx pullfrog init`
|
|
: "timed out waiting for app installation.\n" +
|
|
` ${pc.dim("if your org requires admin approval, ask an admin to approve,")}\n` +
|
|
` ${pc.dim("then re-run:")} npx pullfrog init`
|
|
);
|
|
}
|
|
|
|
// ── secret management ──
|
|
|
|
type StorageMethod = "pullfrog" | "github";
|
|
type SecretScope = "account" | "repo";
|
|
|
|
type SecretSetResult = { saved: boolean; orgFailed: boolean };
|
|
|
|
function setGhSecret(ctx: {
|
|
name: string;
|
|
value: string;
|
|
org: string | null;
|
|
repoSlug: string;
|
|
}): SecretSetResult {
|
|
let orgFailed = false;
|
|
|
|
if (ctx.org) {
|
|
try {
|
|
execFileSync("gh", ["secret", "set", ctx.name, "--org", ctx.org, "--visibility", "all"], {
|
|
input: ctx.value,
|
|
stdio: ["pipe", "ignore", "pipe"],
|
|
encoding: "utf-8",
|
|
});
|
|
return { saved: true, orgFailed: false };
|
|
} catch {
|
|
orgFailed = true;
|
|
}
|
|
}
|
|
|
|
try {
|
|
execFileSync("gh", ["secret", "set", ctx.name, "--repo", ctx.repoSlug], {
|
|
input: ctx.value,
|
|
stdio: ["pipe", "ignore", "pipe"],
|
|
encoding: "utf-8",
|
|
});
|
|
return { saved: true, orgFailed };
|
|
} catch {
|
|
return { saved: false, orgFailed };
|
|
}
|
|
}
|
|
|
|
type PullfrogSecretResult = { saved: boolean; error: string };
|
|
|
|
async function setPullfrogSecret(ctx: {
|
|
token: string;
|
|
owner: string;
|
|
repo: string;
|
|
name: string;
|
|
value: string;
|
|
scope: SecretScope;
|
|
}): Promise<PullfrogSecretResult> {
|
|
const result = await pullfrogApi<{ success?: boolean; error?: string }>({
|
|
path: "/api/cli/secrets",
|
|
token: ctx.token,
|
|
method: "POST",
|
|
body: {
|
|
owner: ctx.owner,
|
|
repo: ctx.repo,
|
|
name: ctx.name,
|
|
value: ctx.value,
|
|
scope: ctx.scope,
|
|
},
|
|
});
|
|
if (result.ok && result.data.success === true) {
|
|
return { saved: true, error: "" };
|
|
}
|
|
return { saved: false, error: result.data.error || `api returned ${result.status}` };
|
|
}
|
|
|
|
async function promptScope(ctx: { owner: string; repo: string }): Promise<SecretScope> {
|
|
const scope = await p.select<SecretScope>({
|
|
message: "secret scope",
|
|
options: [
|
|
{ value: "account", label: `${ctx.owner} organization`, hint: "shared across repos" },
|
|
{ value: "repo", label: `${ctx.owner}/${ctx.repo} only` },
|
|
],
|
|
});
|
|
handleCancel(scope);
|
|
return scope;
|
|
}
|
|
|
|
async function handleSecret(ctx: {
|
|
token: string;
|
|
owner: string;
|
|
repo: string;
|
|
provider: CliProvider;
|
|
secrets: SecretsInfo;
|
|
}): Promise<void> {
|
|
const repoSecretsUrl = `https://github.com/${ctx.owner}/${ctx.repo}/settings/secrets/actions`;
|
|
|
|
const matches: { name: string; source: string }[] = [];
|
|
for (const v of ctx.provider.envVars) {
|
|
if (ctx.secrets.pullfrogSecrets.includes(v)) matches.push({ name: v, source: "pullfrog" });
|
|
else if (ctx.secrets.secretsAccessible && ctx.secrets.orgSecrets.includes(v))
|
|
matches.push({ name: v, source: "org secret" });
|
|
else if (ctx.secrets.secretsAccessible && ctx.secrets.repoSecrets.includes(v))
|
|
matches.push({ name: v, source: "repo secret" });
|
|
}
|
|
|
|
if (matches.length > 0) {
|
|
activeSpin!.start("");
|
|
activeSpin!.stop("secrets already configured");
|
|
for (const m of matches) {
|
|
process.stdout.write(
|
|
`${pc.gray(p.S_BAR)} ${pc.cyan(m.name)} ${pc.dim(`(${m.source})`)}\n`
|
|
);
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (!ctx.secrets.secretsAccessible) {
|
|
p.log.info(`could not verify GitHub secrets (app lacks permission)`);
|
|
}
|
|
|
|
const hasOAuthOption = ctx.provider.envVars.includes("CLAUDE_CODE_OAUTH_TOKEN");
|
|
let envVar = ctx.provider.envVars[0];
|
|
|
|
if (hasOAuthOption) {
|
|
const authMethod = await p.select({
|
|
message: "which credential do you want to use?",
|
|
options: [
|
|
{
|
|
value: "oauth",
|
|
label: "Claude Code OAuth token",
|
|
hint: `run ${pc.cyan("claude setup-token")} — works with Pro/Max subscriptions`,
|
|
},
|
|
{
|
|
value: "api",
|
|
label: "Anthropic API key",
|
|
hint: "from console.anthropic.com",
|
|
},
|
|
],
|
|
});
|
|
handleCancel(authMethod);
|
|
if (authMethod === "oauth") envVar = "CLAUDE_CODE_OAUTH_TOKEN";
|
|
}
|
|
|
|
const method = await p.select<StorageMethod>({
|
|
message: `where should ${pc.cyan(envVar)} be stored?`,
|
|
options: [
|
|
{
|
|
value: "pullfrog",
|
|
label: "Pullfrog",
|
|
hint: "recommended — auto-injected, no workflow changes",
|
|
},
|
|
{
|
|
value: "github",
|
|
label: "GitHub Actions secret",
|
|
hint: "requires env block in pullfrog.yml",
|
|
},
|
|
],
|
|
});
|
|
handleCancel(method);
|
|
|
|
const pasteLabel =
|
|
envVar === "CLAUDE_CODE_OAUTH_TOKEN" ? "OAuth token" : `${ctx.provider.name} API key`;
|
|
const apiKey = await p.password({
|
|
message: `paste your ${pasteLabel} ${pc.dim("(Enter to skip)")}`,
|
|
mask: "*",
|
|
validate: () => undefined,
|
|
});
|
|
handleCancel(apiKey);
|
|
|
|
if (!apiKey) {
|
|
p.log.info(
|
|
`skipped — set it manually at:\n ${pc.dim(method === "pullfrog" ? `${PULLFROG_API_URL}/console/${ctx.owner}` : repoSecretsUrl)}`
|
|
);
|
|
return;
|
|
}
|
|
|
|
if (method === "pullfrog") {
|
|
const scope: SecretScope = ctx.secrets.isOrg ? await promptScope(ctx) : "account";
|
|
|
|
activeSpin!.start(`saving ${envVar}`);
|
|
let saveResult: PullfrogSecretResult;
|
|
try {
|
|
saveResult = await setPullfrogSecret({
|
|
token: ctx.token,
|
|
owner: ctx.owner,
|
|
repo: ctx.repo,
|
|
name: envVar,
|
|
value: apiKey,
|
|
scope,
|
|
});
|
|
} catch (error) {
|
|
activeSpin!.stop(pc.red("could not save secret"));
|
|
p.log.warn(
|
|
`${error instanceof Error ? error.message : "network error"}\n set it manually at: ${pc.dim(`${PULLFROG_API_URL}/console/${ctx.owner}`)}`
|
|
);
|
|
return;
|
|
}
|
|
|
|
if (saveResult.saved) {
|
|
activeSpin!.stop(`saved ${pc.cyan(envVar)} to Pullfrog`);
|
|
} else {
|
|
activeSpin!.stop(pc.red("could not save secret"));
|
|
p.log.warn(
|
|
`${saveResult.error}\n set it manually at: ${pc.dim(`${PULLFROG_API_URL}/console/${ctx.owner}`)}`
|
|
);
|
|
}
|
|
return;
|
|
}
|
|
|
|
// github actions secret path
|
|
let org: string | null = null;
|
|
if (ctx.secrets.isOrg) {
|
|
const scope = await promptScope(ctx);
|
|
org = scope === "account" ? ctx.owner : null;
|
|
}
|
|
|
|
const secretsUrl = org
|
|
? `https://github.com/organizations/${org}/settings/secrets/actions`
|
|
: repoSecretsUrl;
|
|
|
|
activeSpin!.start(`saving ${envVar}`);
|
|
const secretResult = setGhSecret({
|
|
name: envVar,
|
|
value: apiKey,
|
|
org,
|
|
repoSlug: `${ctx.owner}/${ctx.repo}`,
|
|
});
|
|
if (secretResult.saved) {
|
|
activeSpin!.stop(
|
|
`saved ${pc.cyan(envVar)} to ${org && !secretResult.orgFailed ? `${pc.dim(ctx.owner)} org secret` : "GitHub Actions secret"}`
|
|
);
|
|
if (secretResult.orgFailed) {
|
|
p.log.warn("org secret failed (admin access required) — saved as repo secret instead");
|
|
}
|
|
} else {
|
|
activeSpin!.stop(pc.red("could not set secret"));
|
|
p.log.warn(`set it manually at:\n ${pc.dim(secretsUrl)}`);
|
|
}
|
|
}
|
|
|
|
async function promptTestRun(ctx: { token: string; owner: string; repo: string }): Promise<void> {
|
|
const proceed = await p.select({
|
|
message: "test your installation?",
|
|
options: [
|
|
{ value: true, label: "yes", hint: "dispatches a test run in your GitHub Actions" },
|
|
{ value: false, label: "skip" },
|
|
],
|
|
});
|
|
handleCancel(proceed);
|
|
if (!proceed) return;
|
|
|
|
activeSpin!.start("dispatching test run");
|
|
const result = await pullfrogApi<DispatchApiData>({
|
|
path: "/api/cli/dispatch",
|
|
token: ctx.token,
|
|
method: "POST",
|
|
body: { owner: ctx.owner, repo: ctx.repo, prompt: "Tell me a joke" },
|
|
});
|
|
|
|
if (!result.ok) {
|
|
activeSpin!.stop(pc.red("could not dispatch"));
|
|
p.log.warn(result.data.error || `dispatch failed (${result.status})`);
|
|
return;
|
|
}
|
|
|
|
activeSpin!.stop("dispatched test run");
|
|
if (result.data.url) {
|
|
process.stdout.write(
|
|
`${pc.gray(p.S_BAR)} ${link(pc.dim(result.data.url), result.data.url)}\n`
|
|
);
|
|
openBrowser(result.data.url);
|
|
}
|
|
}
|
|
|
|
// ── main ──
|
|
|
|
async function main() {
|
|
p.intro(pc.bgGreen(pc.black(" pullfrog ")));
|
|
|
|
const spin = p.spinner();
|
|
activeSpin = spin;
|
|
|
|
// 1. authenticate
|
|
spin.start("authenticating with github");
|
|
const token = getGhToken();
|
|
const userResult = await ghApi<{ login: string }>("/user", token);
|
|
const user = userResult.data;
|
|
|
|
// gho_ tokens from `gh auth login` expose scopes via x-oauth-scopes header.
|
|
// fine-grained PATs (github_pat_) don't return scopes — they pass this check.
|
|
// split on ", " and match exact scope — .includes("repo") would false-positive on "public_repo"
|
|
const scopeSet = userResult.scopes !== null ? new Set(userResult.scopes.split(", ")) : null;
|
|
if (scopeSet !== null && !scopeSet.has("repo")) {
|
|
bail(
|
|
`your token is missing the ${pc.bold('"repo"')} scope.\n` +
|
|
` ${pc.dim("run:")} gh auth refresh --scopes repo\n` +
|
|
` ${pc.dim("then:")} npx pullfrog init`
|
|
);
|
|
}
|
|
|
|
spin.stop(`hello, ${pc.cyan(`@${user.login}`)}`);
|
|
|
|
// 2. detect repo
|
|
spin.start("detecting repository");
|
|
const remote = parseGitRemote();
|
|
spin.stop(`detected repo ${pc.cyan(`${remote.owner}/${remote.repo}`)}`);
|
|
|
|
// 3. ensure app installation + check secrets
|
|
const secrets = await ensureInstallation({ token, owner: remote.owner, repo: remote.repo });
|
|
|
|
// 4. select provider + model (skip if already set)
|
|
let model: string;
|
|
let provider: CliProvider;
|
|
|
|
if (secrets.model) {
|
|
model = secrets.model;
|
|
const resolved = resolveModelProvider(secrets.model);
|
|
if (!resolved) bail(`unknown model provider: ${secrets.model}`);
|
|
provider = resolved;
|
|
// walk the fallback chain so a deprecated stored slug shows the model
|
|
// the run will actually execute against (e.g. "GPT", not "GPT Codex").
|
|
const displayAlias = resolveDisplayAlias(secrets.model);
|
|
const label = displayAlias ? displayAlias.displayName : secrets.model;
|
|
spin.start("");
|
|
spin.stop(`using model ${pc.cyan(label)}`);
|
|
} else {
|
|
const providerId = await p.select({
|
|
message: "select your preferred model provider",
|
|
options: CLI_PROVIDERS.map((cp) => ({
|
|
value: cp.id,
|
|
label: cp.name,
|
|
})),
|
|
});
|
|
handleCancel(providerId);
|
|
|
|
const found = CLI_PROVIDERS.find((cp) => cp.id === providerId);
|
|
if (!found) bail(`unknown provider: ${providerId}`);
|
|
provider = found;
|
|
|
|
if (provider.models.length === 1) {
|
|
model = provider.models[0].value;
|
|
spin.start("");
|
|
spin.stop(`using ${pc.bold(provider.models[0].label)}`);
|
|
} else {
|
|
const recommendedModel = provider.models.find((m) => m.hint === "recommended");
|
|
const options = provider.models.map((m) => {
|
|
if (m.hint) return { value: m.value, label: m.label, hint: m.hint };
|
|
return { value: m.value, label: m.label };
|
|
});
|
|
const selected = await p.select(
|
|
recommendedModel
|
|
? { message: "select model", initialValue: recommendedModel.value, options }
|
|
: { message: "select model", options }
|
|
);
|
|
handleCancel(selected);
|
|
model = selected;
|
|
}
|
|
}
|
|
|
|
// 5. check/set secret
|
|
await handleSecret({ token, owner: remote.owner, repo: remote.repo, provider, secrets });
|
|
|
|
// 6. create workflow
|
|
spin.start("creating pullfrog.yml workflow");
|
|
|
|
const result = await pullfrogApi<SetupApiData>({
|
|
path: "/api/cli/setup",
|
|
token,
|
|
method: "POST",
|
|
body: { owner: remote.owner, repo: remote.repo, model },
|
|
});
|
|
|
|
if (!result.ok) {
|
|
bail(result.data.error || `api returned ${result.status}`);
|
|
}
|
|
|
|
let skipTestRun = false;
|
|
|
|
if (result.data.already_existed) {
|
|
spin.stop("pullfrog.yml already exists");
|
|
} else if (result.data.pull_request_url) {
|
|
spin.stop("opened pull request with pullfrog.yml");
|
|
process.stdout.write(
|
|
`${pc.gray(p.S_BAR)} ${link(pc.dim(result.data.pull_request_url), result.data.pull_request_url)}\n`
|
|
);
|
|
openBrowser(result.data.pull_request_url);
|
|
|
|
const merged = await p.select({
|
|
message: "merge the PR to activate pullfrog, then continue",
|
|
options: [
|
|
{ value: true, label: "continue", hint: "PR has been merged" },
|
|
{ value: false, label: "skip" },
|
|
],
|
|
});
|
|
handleCancel(merged);
|
|
if (!merged) skipTestRun = true;
|
|
} else {
|
|
const short = result.data.hash?.slice(0, 7);
|
|
spin.stop(
|
|
short ? `committed pullfrog.yml to repo ${pc.dim(short)}` : "committed pullfrog.yml to repo"
|
|
);
|
|
}
|
|
|
|
if (!skipTestRun && !secrets.hasRuns) {
|
|
await promptTestRun({ token, owner: remote.owner, repo: remote.repo });
|
|
}
|
|
|
|
const consoleUrl = `${PULLFROG_API_URL}/console/${remote.owner}/${remote.repo}`;
|
|
spin.start("");
|
|
spin.stop("repo is configurable via the Pullfrog dashboard");
|
|
process.stdout.write(`${pc.gray(p.S_BAR)} ${link(pc.dim(consoleUrl), consoleUrl)}\n`);
|
|
activeSpin = null;
|
|
p.outro("done.");
|
|
}
|
|
|
|
interface InitCliParams {
|
|
args: string[];
|
|
prog: string;
|
|
showHelp?: boolean;
|
|
}
|
|
|
|
function printInitUsage(params: { stream: typeof console.log; prog: string }): void {
|
|
params.stream(`usage: ${params.prog} init\n`);
|
|
params.stream("set up pullfrog on the current repository.");
|
|
params.stream("");
|
|
params.stream("options:");
|
|
params.stream(" -h, --help show help");
|
|
}
|
|
|
|
function parseInitArgs(args: string[]) {
|
|
return arg(
|
|
{
|
|
"--help": Boolean,
|
|
"-h": "--help",
|
|
},
|
|
{
|
|
argv: args,
|
|
}
|
|
);
|
|
}
|
|
|
|
export async function runCli(params: InitCliParams): Promise<void> {
|
|
if (params.showHelp) {
|
|
printInitUsage({ stream: console.log, prog: params.prog });
|
|
return;
|
|
}
|
|
|
|
let parsed: ReturnType<typeof parseInitArgs>;
|
|
try {
|
|
parsed = parseInitArgs(params.args);
|
|
} catch (error) {
|
|
const message = error instanceof Error ? error.message : String(error);
|
|
console.error(`${message}\n`);
|
|
printInitUsage({ stream: console.error, prog: params.prog });
|
|
process.exit(1);
|
|
}
|
|
|
|
if (parsed["--help"]) {
|
|
printInitUsage({ stream: console.log, prog: params.prog });
|
|
return;
|
|
}
|
|
|
|
if (parsed._.length > 0) {
|
|
console.error(`unexpected positional arguments for init: ${parsed._.join(" ")}\n`);
|
|
printInitUsage({ stream: console.error, prog: params.prog });
|
|
process.exit(1);
|
|
}
|
|
|
|
await run();
|
|
}
|
|
|
|
export async function run() {
|
|
try {
|
|
await main();
|
|
} catch (error) {
|
|
if (activeSpin) {
|
|
activeSpin.stop(pc.red("failed"));
|
|
activeSpin = null;
|
|
}
|
|
const msg =
|
|
error instanceof Error && error.name === "AbortError"
|
|
? "request timed out — check your network connection and try again"
|
|
: error instanceof Error
|
|
? error.message
|
|
: String(error);
|
|
p.log.error(msg);
|
|
process.exit(1);
|
|
}
|
|
}
|