Agent & model refactor (#478)

* agent & model refactor with ASKPASS git auth, UI restructure, clerk v7 Made-with: Cursor * fix stale agent/effort refs, add tests for askpass + model resolution - reviewCleanup.ts: payload.agent -> payload.model, remove effort - selectMode.ts PlanEdit: remove delegation/subagent/effort references - pullfrog.yml.ts: update env vars (drop GOOGLE_API_KEY/CURSOR_API_KEY, add GOOGLE_GENERATIVE_AI_API_KEY/XAI_API_KEY/MOONSHOT_API_KEY/OPENCODE_API_KEY) - FlagsSettings/RepoInstructionsSection: remove stale effort/timeout copy - new: gitAuthServer.test.ts (10 tests — lifecycle, token delivery, tamper detection, script gen) - new: agent.test.ts (4 tests — default opentoad, AGENT_OVERRIDE, invalid override) - new: models.test.ts (19 tests — parseModel, resolution, registry invariants) - update models.dev snapshot Made-with: Cursor * fix changed-agents.sh to filter legacy agent files from CI matrix legacy agent files (claude.ts, codex.ts, etc.) are @ts-nocheck and not exported from index.ts. changed-agents.sh now reads index.ts imports to build the active agent set and treats changes to inactive files as non-agent changes (opentoad canary only). Made-with: Cursor * remove MCP file tools, old agent harnesses, and obsolete security tests ASKPASS-based git auth makes the old MCP file tool security layer unnecessary: - token never in subprocess env, so symlink/gitattributes/hook attacks can't exfiltrate it - agents now use native file tools (OpenCode builtin read/edit) deleted: - action/mcp/file.ts (file_read, file_write, file_edit, file_delete, list_directory) - action/mcp/index.ts (dead re-export) - agent harnesses: claude.ts, codex.ts, cursor.ts, gemini.ts, opencode.ts - opencode-runner.ts (inlined into opentoad.ts) - security tests that validated MCP file tool restrictions - commented-out three-step review flow (~300 lines) - sanitizeSchema/wrapSchema dead code from mcp/shared.ts - OPENCODE_MODEL_MINI/MAX env vars (effort-level model overrides removed) updated test prompts to use generic file ops instead of MCP tool names. restored pkg-json-scripts + requirements-txt-attack (test --ignore-scripts defense). Made-with: Cursor * bump actions/checkout v4 → v6 (node 24) node 20 actions deprecated june 2, 2026. Made-with: Cursor * temporarily disable fail-fast on agnostic tests to debug checkout@v6 Made-with: Cursor * re-enable fail-fast on agnostic tests Made-with: Cursor * fix test token mismatch: mint OIDC tokens scoped to target repo CI tests override GITHUB_REPOSITORY to pullfrog/test-repo but inherit the runner's GITHUB_TOKEN (scoped to pullfrog/app), causing 401s on every run-context fetch. Clear GITHUB_TOKEN in the test subprocess so ensureGitHubToken() mints a properly scoped token via OIDC. Also centralizes the default GITHUB_REPOSITORY in runAgentStreaming instead of repeating it in every test file, and fixes preview-cleanup to remove workers from all queues (not just name-matching ones). Made-with: Cursor * fix ensureGitHubToken to try OIDC when app credentials are absent ensureGitHubToken only attempted token minting when GITHUB_APP_ID and GITHUB_PRIVATE_KEY were set. In CI, OIDC is available but app creds aren't exposed — so the guard prevented minting entirely. Made-with: Cursor * dead code cleanup: remove remnants of deleted agents, file tools, effort system remove unused @anthropic-ai/claude-agent-sdk and @openai/codex-sdk deps, orphaned file-tool security tests, dead GEMINI_MODEL passthrough, stale opencode-runner wiki refs, deleted test file references, and MCP file tool docs. rename docs/effort → docs/models. fix vitest setup: move dotenv to globalSetup (runs once before forks instead of per-file, 19s → 200ms). Made-with: Cursor * address review feedback: remove dead code, update stale references - remove AGENT_OVERRIDE (only opentoad exists) - remove shellToolName plumbing (always restricted shell) - bump action version to 0.0.179 - remove CURSOR_API_KEY from all workflows/configs - remove OPENCODE_MODEL_MINI/MAX from workflows/docs - delete wiki/effort.md, rewrite docs/effort.mdx as "Models" - rewrite wiki/modes.md: orchestrator/subagent → single agent - simplify flag system: drop builtin flag extraction (debug, effort, timeout, agent), keep custom flag replacement only - reserve all legacy flag names to prevent custom flag conflicts Made-with: Cursor * regenerate lockfile after removing claude-agent-sdk and codex-sdk Made-with: Cursor * fix import ordering, add lockfile check to pre-push hook Made-with: Cursor * remove dead debug payload field, stale packageExtensions Made-with: Cursor * merge proc-sandbox and token-exfil into a single test proc-sandbox and token-exfil were duplicative — both tested that SANDBOX_TEST_TOKEN couldn't be exfiltrated. consolidated into token-exfil with shell:restricted (which actually exercises filterEnv) and the /proc attack vector hints from proc-sandbox. Made-with: Cursor * fix wiki adversarial.md to match actual tokenExfil validator Made-with: Cursor
2026-03-12 05:22:51 +00:00
parent 5bcfae990a
commit 6d25adfd1a
111 changed files with 4196 additions and 10202 deletions
@@ -0,0 +1,138 @@
+import { existsSync, mkdtempSync, readFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { afterEach, describe, expect, it } from "vitest";
+import { type GitAuthServer, startGitAuthServer } from "./gitAuthServer.ts";
+
+let server: GitAuthServer | undefined;
+
+afterEach(async () => {
+  if (server) {
+    await server.close();
+    server = undefined;
+  }
+});
+
+function makeTmpdir(): string {
+  return mkdtempSync(join(tmpdir(), "askpass-test-"));
+}
+
+describe("git auth server lifecycle", () => {
+  it("starts and listens on a port", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    expect(server.port).toBeGreaterThan(0);
+  });
+
+  it("closes cleanly", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const port = server.port;
+    await server.close();
+    server = undefined;
+
+    // port should no longer accept connections
+    const err = await fetch(`http://127.0.0.1:${port}/test`).catch((e) => e);
+    expect(err).toBeInstanceOf(Error);
+  });
+});
+
+describe("token delivery", () => {
+  it("returns token on first request with valid code", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code = server.register("ghs_test_token_12345");
+
+    const res = await fetch(`http://127.0.0.1:${server.port}/${code}`);
+    expect(res.status).toBe(200);
+    const body = await res.text();
+    expect(body).toBe("ghs_test_token_12345");
+  });
+
+  it("returns 404 for unknown code", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+
+    const res = await fetch(`http://127.0.0.1:${server.port}/nonexistent-code`);
+    expect(res.status).toBe(404);
+  });
+
+  it("returns 400 for empty code", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+
+    const res = await fetch(`http://127.0.0.1:${server.port}/`);
+    expect(res.status).toBe(400);
+  });
+
+  it("returns 405 for non-GET methods", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code = server.register("token");
+
+    const res = await fetch(`http://127.0.0.1:${server.port}/${code}`, { method: "POST" });
+    expect(res.status).toBe(405);
+  });
+});
+
+describe("single-use enforcement (tamper detection)", () => {
+  it("returns 409 on second use of same code", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code = server.register("ghs_tamper_test");
+
+    const first = await fetch(`http://127.0.0.1:${server.port}/${code}`);
+    expect(first.status).toBe(200);
+
+    const second = await fetch(`http://127.0.0.1:${server.port}/${code}`);
+    expect(second.status).toBe(409);
+    const body = await second.text();
+    expect(body).toBe("compromised");
+  });
+
+  it("each register() call produces an independent code", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code1 = server.register("token-a");
+    const code2 = server.register("token-b");
+
+    expect(code1).not.toBe(code2);
+
+    const res1 = await fetch(`http://127.0.0.1:${server.port}/${code1}`);
+    expect(await res1.text()).toBe("token-a");
+
+    const res2 = await fetch(`http://127.0.0.1:${server.port}/${code2}`);
+    expect(await res2.text()).toBe("token-b");
+  });
+});
+
+describe("askpass script generation", () => {
+  it("writes an executable script file", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code = server.register("ghs_script_test");
+    const scriptPath = server.writeAskpassScript(code);
+
+    expect(existsSync(scriptPath)).toBe(true);
+    expect(scriptPath.startsWith(tmp)).toBe(true);
+
+    const content = readFileSync(scriptPath, "utf-8");
+    expect(content).toContain("#!/usr/bin/env node");
+    expect(content).toContain(String(server.port));
+    expect(content).toContain(code);
+    // token should NOT be in the script — only port and code
+    expect(content).not.toContain("ghs_script_test");
+  });
+
+  it("script handles Username prompt locally (no server call)", async () => {
+    const tmp = makeTmpdir();
+    server = await startGitAuthServer(tmp);
+    const code = server.register("ghs_username_test");
+    const scriptPath = server.writeAskpassScript(code);
+    const content = readFileSync(scriptPath, "utf-8");
+
+    // script checks for /^Username/i and returns "x-access-token" without HTTP
+    expect(content).toContain("Username");
+    expect(content).toContain("x-access-token");
+  });
+});