1de60d74fdb00a5f30ef6e4053b30d61355be811
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 David Blass
|
d3b5340583 |
fix: audit batch — MCP timeouts, entryPost, vip_audit 404s, and 6 more (#824)
* fix: 9 unaddressed log-audit / run-audit findings Co-authored-by: Cursor <cursoragent@cursor.com> #815 entryPost stdlib-only imports; #823 MCP timeoutMs on checkout_pr/shell; #816 FREE_FALLBACK → opencode/big-pickle; #822 chunk GraphQL nodes ≤100; #817/#821 vip_audit 404 skip paths; #813 longer serializable retries; #818 run-context handler-entered log; #805 audit severity template. * fix: update footer test for big-pickle fallback slug Co-authored-by: Cursor <cursoragent@cursor.com> * fix: anneal round 1 — ghaCore getState casing, post-hook timeout Match @actions/core STATE_ key semantics (no uppercasing), cap postApiFetch at 30s, trim serializable retries to stay under GitHub's 10s webhook window, log Clerk failures in getUserTokenByGithubLogin. Co-authored-by: Cursor <cursoragent@cursor.com> * revert: drop run-context handler log (#818 deferred) The #692 client-side fix is already on main; residual SyntaxError hits are old action pins. Per-request log added noise without fixing anything. Co-authored-by: Cursor <cursoragent@cursor.com> * document per-issue Closes syntax for audit PRs GitHub only auto-closes the first issue when numbers are comma-separated; /audits and AGENTS.md now require Closes before each issue number. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: anneal round 2 — outreach privacy, alert resilience, vertex cleanup Filter private repos from VIP authority output, harden console alert lines against DB failures, drop spoofable changesets body check, and unset GOOGLE_APPLICATION_CREDENTIALS after vertex credential cleanup. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor: drop codexHome re-export of detectCodexRefresh Import detectCodexRefresh directly from codexRefreshDetect.ts everywhere; rename the unit test file to match. codexHome.ts stays install-only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: drop deprecated minimax-m2.5-free; add paid minimax-m2.5 Remove the deprecated free MiniMax promo from the catalog, docs, and tests. BYOK fallback and picker copy stay on opencode/big-pickle. Add opencode/minimax-m2.5 and openrouter/minimax-m2.5 for Zen BYOK and Router. Pin #816 regressions with freeFallbackCatalog and runErrorRenderer unit tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: hidden minimax-m2.5-free fallback for stored slugs Re-add opencode/minimax-m2.5-free as a hidden fallback alias to big-pickle so repos with the legacy slug still resolve as free. Drop live Zen API experiment tests in freeFallbackCatalog.test.ts. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Colin McDonnell
|
c0988e35b0 |
fix(security): block docker socket from sandboxed shell; disable opencode batch_tool
two real CI failures on main, both shipping bugs in the action: 1. `token-exfil-claude` was a real sandbox escape: GHA `ubuntu-latest` puts `runner` in the `docker` group, so a sandboxed shell could run `docker run --pid=host --privileged busybox cat /proc/<parent>/environ` and read the action process's env (which holds user secrets) — fully bypassing the unshare PID-namespace. fix: inside the sandbox's mount namespace (already private via `--mount-proc` which implies `--mount`), bind-mount /dev/null over /var/run/docker.sock (+ podman/containerd/crio variants) so any container-runtime socket connect from the sandbox fails. only affects sandboxed shells — host runner mount table is untouched, so user workflow steps outside pullfrog keep working. 2. `restricted-opencode` regressed in #719 (`experimental.batch_tool`). opencode's batch tool rejects MCP tools with `"Tool '<name>' not in registry. External tools (MCP, environment) cannot be batched."` when a model emits parallel `pullfrog_shell` (or any MCP) tool_use blocks, opencode internally routes them through batch, they all fail, the model misreads the error as "the tool doesn't exist", and gives up. caught by a `lens:` subagent in the restricted test concluding shell was unavailable and setting `DIAGNOSTIC_ID=empty`. drop `batch_tool: true` and the matching opencode-specific guidance in `instructions.ts` — native parallel tool_use (multiple tool_use blocks per assistant message) still works for both built-in and MCP tools without batch, so we lose only the 1-25 wrapper, not parallelism. |
||
|
Colin McDonnell
|
2960d51493 |
shell tool: cap output at 5K chars and spill overflow to tempfile (#732)
unbounded shell tool output blows the agent's context window on commands
that dump big logs (test runners, build tools, grep on large trees). cap
the inline body at 5000 chars; on overflow, persist the full output to
${PULLFROG_TEMP_DIR}/shell-<id>.log and return the tail prefixed with a
sentinel pointing at the saved path. agents re-read the tempfile with
cat/tail/grep when they need more.
|
||
|
Colin McDonnell
|
b8ac42e875 |
mcp: embed example calls in top-level tool descriptions (#723)
* mcp: embed example calls in top-level tool descriptions
agents (esp. claude sonnet) hallucinate param names from training-data
priors — `pr_number` instead of `pull_number`, `summary` instead of
`body`, full subcommand strings jammed into `git({command})` like it
were `shell({command})`. each error burns a tool round-trip plus a
follow-up ToolSearch, ~40+ events / 24h, no observable recovery cost
to us but visible to users in agent logs.
cheapest fix: add a sample formatted function call to every affected
tool's top-level description. example anchors are more reliable than
schema descriptions alone because the model treats descriptions as
narrative but call examples as canonical structure. for `git` and
`shell` (whose `command` fields collide), include explicit
counter-examples disambiguating which tool owns which shape.
no schema aliases / coercion yet — try the cheap thing first; if the
next audit window still shows the same hallucination rate, layer
aliases on top per #585's recommendation.
closes #585, closes #701
* mcp: drop negative anchors from tool descriptions
negation is a footgun in tool descriptions — telling the model "NOT
pr_number" makes pr_number more salient, not less. let the positive
example carry the schema and trust the model to read it.
removes:
- "the parameter is pull_number (a number), NOT pr_number" and
similar across checkout_pr, get_pull_request, list_pull_request_reviews,
get_review_comments, create_pull_request_review
- "NOT summary, message, or content" on report_progress
- "WRONG: git({ command: 'log --oneline' })" counter-example on git
- redundant param-type restatements after the example (e.g. "depth is a
number, not a string" on git_fetch, "description is required" on shell)
keeps a single positive example per tool. for tools with multiple call
shapes (git, git_fetch, push_branch), two positive examples instead of
one + a counter-example.
|
||
|
David Blass
|
5518890b18 |
learnings: TOC + section taxonomy + 100k cap, hygiene rules, tool-quirk descriptions (#717)
* audit learnings: reshape reflection prompt + bake tool quirks into descriptions (#619) Cross-repo audit of the 48 repos with non-null learnings turned up two recurring failure modes: 1. ~25-30% of bullets across the most-active repos are pullfrog-tool quirks ("shell timeout is in milliseconds", "git args must be a JSON array", "create_pull_request_review drops out-of-hunk comments", "push_branch may report timeout when push succeeded", "checkout_pr shallow.lock retries", "commit_id needs full 40-char SHA"). These are universal across repos and should live in tool descriptions, not be rediscovered and stored 48 times. Tool descriptions now surface them. 2. Bullets are routinely 200-1000 chars (paragraph-length), and 12 of 48 repos are at the 10k cap. The reflection prompt now: caps bullets at ~240 chars (one specific fact), bans PR/review/commit/date-anchored facts that decay within weeks, bans tool-quirk learnings, and tells the agent that cap pressure means compress+prune existing bullets, not skip new findings. Co-authored-by: Cursor <cursoragent@cursor.com> * learnings: add server-generated TOC, fixed section taxonomy, raise cap to 100k (#707) Cap goes 10k → 100k. Reads stay bounded because the seeded file now opens with a server-generated table of contents listing every `## ` section's line range — agents read the TOC, then `read_file offset/limit` just the sections relevant to the current task instead of slurping the whole file. ## Section taxonomy (fixed) `## Build & test`, `## CI`, `## Conventions`, `## Architecture`, `## Gotchas`. Free-form `### ` sub-headings inside a section are fine. Pre-taxonomy free-text rows get wrapped in a `## Legacy` carve-out on first seed so they remain visible while the agent gradually re-curates them during reflection turns. ## Storage shape unchanged `Repo.learnings` still holds raw markdown (no schema migration). The TOC is a pure read-side affordance: prepended at seed time, stripped from the agent-edited file before persist. Markers `<!-- pullfrog-learnings-toc:* -->` delimit the strip region. Agent edits inside the markers are discarded. ## Round-trip semantics `seedLearningsFile` now returns `{ path, canonicalSeed }` where `canonicalSeed` is the post-TOC body — same shape `readLearningsFile` returns at end-of-run, so `persistLearnings` byte-compares them directly to skip the no-op PATCH. Empty-repo first runs end up with the section scaffold both as seed and as read-back, so untouched runs still short-circuit cleanly. ## Reflection prompt Adds explicit section-placement guidance (place each new bullet under the most relevant `## `; do NOT add new top-level headings; do NOT edit anything between the TOC markers). Carries forward the bullet hygiene from the previous commit: ≤240 chars per bullet, no pullfrog-tool quirks (those belong in tool descriptions), no PR/review/commit/date references. The "near cap" framing is replaced with "compress and prune within a section when it grows noisy" since the cap pressure that drove cramming is gone. Co-authored-by: Cursor <cursoragent@cursor.com> * anneal round 1: line-anchored taxonomy detect, partial-merge, line-boundary truncation, scaffold-empty UI Multi-lens review of the TOC + taxonomy diff surfaced a cluster of correctness and operational bugs. Fixes: - `hasAnyTaxonomyHeading` used `String.includes("## X")` which false-positives on `### X` (the `## ` substring sits inside `### `), prose containing `## CI`, fenced code documenting markdown, etc. Replaced with a line-anchored predicate that reuses `parseHeadings` so detection and TOC construction stay consistent. - The "any heading present → pass through verbatim" rule meant a body with one taxonomy heading would seed without the other four. Worse, requiring all five would flip a body back into Legacy when the agent legitimately pruned a section to empty. New `partial` kind: keep existing content in place, append missing sections in canonical order so the agent always has the full scaffold without losing pruning intent. - `stripLearningsToc` collapsed `\n{3,}` globally; `canonicalSeed` doesn't, so an untouched body with intentional triple-newline spacing would compare unequal and burn a spurious LearningsRevision row each run. Drop the global collapse — only the leading newlines that the strip itself introduces are normalized. - 100k truncation via `slice(0, 100_000)` could cut mid-line, breaking `parseHeadings` (whole-line `^## `) on the next seed and flipping a cut body back into Legacy. New `truncateAtLineBoundary` cuts at the last newline before the cap. - `LearningsSection.tsx` rendered a scaffold-only body as "has learnings" instead of the empty placeholder. Added a `hasOnlyEmptyScaffold` guard so the console behaves the same as pre-PR for the empty case. - Seed log line distinguishes `kind=structured/partial/legacy-wrapped/ empty` instead of `existing=yes/no`, so operators can spot legacy migration activity in logs. - New tests cover: substring false-positive (`### Build & test`, in-prose mentions), partial-taxonomy merge (no Legacy wrap), full-taxonomy structured pass-through, last-newline truncation, triple-newline preservation. Deferred (documented in PR body): deploy-ordering footgun (action before API), rollback for rows >10k, Gemini sanitizer dropping `description` on `anyOf` branches, reflection-on-failed-runs. Co-authored-by: Cursor <cursoragent@cursor.com> * anneal r2: hard-truncate fallback when line boundary discards >4k Round-2 review caught a regression in `truncateAtLineBoundary`: when the only newline within the first 100k chars sits near the start (e.g. one heading + 100k+ char single line — pathological pasted log dumps), the line-boundary cut discards almost all of the body. losing one partial line is preferable to losing kilobytes; threshold the fallback at 4k. Co-authored-by: Cursor <cursoragent@cursor.com> * move TOC out of file: prompt-side rendering, server-parsed headings drops the in-file TOC + fixed taxonomy in favor of: - file on disk = verbatim Repo.learnings (no markers, no scaffold) - server parses headings (mdast-util-from-markdown) at run-context time and returns them as RepoSettings.learningsHeadings - action renders heading TOC into the LEARNINGS prompt section as parenthesized line ranges like `Build & test (L1-L42)` with hierarchy via 2-space indent off the shallowest depth - reflection prompt teaches agent-curated structure with a soft 300-line per-section cap and explicit guidance to restructure flat legacy lists cuts 8 helpers (ensureSections, stripLearningsToc, assembleFile, buildTocBlock, parseHeadings, buildSectionScaffold, hasAnyTaxonomyHeading, LEARNINGS_SECTIONS) and the canonicalSeed round-trip dance. action seedLearningsFile is now { path } only; main.ts byte-compares the trimmed read-back against (current ?? "").trim() to gate the persist PATCH. truncateAtLineBoundary kept for safety. new tests: - test/learningsToc.test.ts (11 parser cases incl. fenced-code, blockquote, arbitrary h1-h6 nesting, startLine-points-at-heading invariant) - action/utils/learningsTocRender.test.ts (7 renderer cases) --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Colin McDonnell <colinmcd94@gmail.com> |
||
|
Colin McDonnell
|
2799cce4bf |
homepage redesign + docs cleanup + agent prompting (#540)
* homepage redesign + docs cleanup + agent prompting improvements - rewrote hero section: new tagline, responsive font sizing with clamp(), extracted shared constants for copy management - added feature screenshots (shell isolation, github permissions, mcp tools, agent browser) and ensured consistent image sizing - reworked feature section mobile layout: caption-style descriptions, bigger h3s, image padding - made CTA buttons visible on all breakpoints (stacked on mobile, row on md+) - reorganized docs/tools.mdx into single table with category dividers, simplified tool descriptions - added markdown image syntax instruction to agent system prompt - fixed InfoPopover overflow on small screens - misc: InlineCode proportional sizing, OnboardingCard updates, shell/security doc improvements Made-with: Cursor * update pnpm-lock.yaml for agent-browser 0.25.4 Made-with: Cursor |
||
|
Mateusz Burzyński
|
c0f6f9ef2a |
Browser skill (#485)
* Add `BrowserTool` * add some logging * go with npm install -g * remove dep changes since the switch to npm install -g * tweak * tweak * tweak * tweak * tweak timeout * tweak * remove logs * skill investigation doc * wip * wip * tweak * lock agent-browser version * tweak * logs * logs * more logs * more debug stuff * try this * try this * try this * fix PATH * try this * tweak * tweak * tweak * update wiki entries * update wiki once again * lint fix --------- Co-authored-by: Colin McDonnell <colinmcd94@gmail.com> |
||
|
Colin McDonnell
|
6d25adfd1a |
Agent & model refactor (#478)
* agent & model refactor with ASKPASS git auth, UI restructure, clerk v7 Made-with: Cursor * fix stale agent/effort refs, add tests for askpass + model resolution - reviewCleanup.ts: payload.agent -> payload.model, remove effort - selectMode.ts PlanEdit: remove delegation/subagent/effort references - pullfrog.yml.ts: update env vars (drop GOOGLE_API_KEY/CURSOR_API_KEY, add GOOGLE_GENERATIVE_AI_API_KEY/XAI_API_KEY/MOONSHOT_API_KEY/OPENCODE_API_KEY) - FlagsSettings/RepoInstructionsSection: remove stale effort/timeout copy - new: gitAuthServer.test.ts (10 tests — lifecycle, token delivery, tamper detection, script gen) - new: agent.test.ts (4 tests — default opentoad, AGENT_OVERRIDE, invalid override) - new: models.test.ts (19 tests — parseModel, resolution, registry invariants) - update models.dev snapshot Made-with: Cursor * fix changed-agents.sh to filter legacy agent files from CI matrix legacy agent files (claude.ts, codex.ts, etc.) are @ts-nocheck and not exported from index.ts. changed-agents.sh now reads index.ts imports to build the active agent set and treats changes to inactive files as non-agent changes (opentoad canary only). Made-with: Cursor * remove MCP file tools, old agent harnesses, and obsolete security tests ASKPASS-based git auth makes the old MCP file tool security layer unnecessary: - token never in subprocess env, so symlink/gitattributes/hook attacks can't exfiltrate it - agents now use native file tools (OpenCode builtin read/edit) deleted: - action/mcp/file.ts (file_read, file_write, file_edit, file_delete, list_directory) - action/mcp/index.ts (dead re-export) - agent harnesses: claude.ts, codex.ts, cursor.ts, gemini.ts, opencode.ts - opencode-runner.ts (inlined into opentoad.ts) - security tests that validated MCP file tool restrictions - commented-out three-step review flow (~300 lines) - sanitizeSchema/wrapSchema dead code from mcp/shared.ts - OPENCODE_MODEL_MINI/MAX env vars (effort-level model overrides removed) updated test prompts to use generic file ops instead of MCP tool names. restored pkg-json-scripts + requirements-txt-attack (test --ignore-scripts defense). Made-with: Cursor * bump actions/checkout v4 → v6 (node 24) node 20 actions deprecated june 2, 2026. Made-with: Cursor * temporarily disable fail-fast on agnostic tests to debug checkout@v6 Made-with: Cursor * re-enable fail-fast on agnostic tests Made-with: Cursor * fix test token mismatch: mint OIDC tokens scoped to target repo CI tests override GITHUB_REPOSITORY to pullfrog/test-repo but inherit the runner's GITHUB_TOKEN (scoped to pullfrog/app), causing 401s on every run-context fetch. Clear GITHUB_TOKEN in the test subprocess so ensureGitHubToken() mints a properly scoped token via OIDC. Also centralizes the default GITHUB_REPOSITORY in runAgentStreaming instead of repeating it in every test file, and fixes preview-cleanup to remove workers from all queues (not just name-matching ones). Made-with: Cursor * fix ensureGitHubToken to try OIDC when app credentials are absent ensureGitHubToken only attempted token minting when GITHUB_APP_ID and GITHUB_PRIVATE_KEY were set. In CI, OIDC is available but app creds aren't exposed — so the guard prevented minting entirely. Made-with: Cursor * dead code cleanup: remove remnants of deleted agents, file tools, effort system remove unused @anthropic-ai/claude-agent-sdk and @openai/codex-sdk deps, orphaned file-tool security tests, dead GEMINI_MODEL passthrough, stale opencode-runner wiki refs, deleted test file references, and MCP file tool docs. rename docs/effort → docs/models. fix vitest setup: move dotenv to globalSetup (runs once before forks instead of per-file, 19s → 200ms). Made-with: Cursor * address review feedback: remove dead code, update stale references - remove AGENT_OVERRIDE (only opentoad exists) - remove shellToolName plumbing (always restricted shell) - bump action version to 0.0.179 - remove CURSOR_API_KEY from all workflows/configs - remove OPENCODE_MODEL_MINI/MAX from workflows/docs - delete wiki/effort.md, rewrite docs/effort.mdx as "Models" - rewrite wiki/modes.md: orchestrator/subagent → single agent - simplify flag system: drop builtin flag extraction (debug, effort, timeout, agent), keep custom flag replacement only - reserve all legacy flag names to prevent custom flag conflicts Made-with: Cursor * regenerate lockfile after removing claude-agent-sdk and codex-sdk Made-with: Cursor * fix import ordering, add lockfile check to pre-push hook Made-with: Cursor * remove dead debug payload field, stale packageExtensions Made-with: Cursor * merge proc-sandbox and token-exfil into a single test proc-sandbox and token-exfil were duplicative — both tested that SANDBOX_TEST_TOKEN couldn't be exfiltrated. consolidated into token-exfil with shell:restricted (which actually exercises filterEnv) and the /proc attack vector hints from proc-sandbox. Made-with: Cursor * fix wiki adversarial.md to match actual tokenExfil validator Made-with: Cursor |
||
|
Colin McDonnell
|
c4d66bf7f6 |
fix: block git in shell tool, add actionable errors for push rejections (#397)
Agents were bypassing the git auth boundary by running `git pull/push` through the shell tool (which has no credentials). This caused auth failures and cascading issues (botched rebases, corrupted files). - shell: block all git commands with error directing to dedicated tools - push_branch: catch "fetch first" rejections with step-by-step recovery - git tool: improve auth redirect errors with specific tool guidance Made-with: Cursor |
||
|
Colin McDonnell
|
f8a871f723 |
fix sudo-unshare sandbox: drop privileges after PROC_CLEANUP (#383)
the sudo-unshare sandbox path runs the entire command as root, causing files modified by shell commands (e.g. git merge) to become root-owned. this breaks file_write/file_edit which run in the Node.js parent process as the normal user (EACCES errors). after PROC_CLEANUP (which needs root for umount/mount), drop back to the original user via `su -p` so file operations match the uid of the parent process. security-neutral: PID namespace isolation is the barrier, not privilege level inside it. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Colin McDonnell
|
4f1e4a2e7a |
fix formatting in shell.ts
Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Colin McDonnell
|
73836d9c8f |
harden proc isolation and filter cross-fork check suite PRs
shell sandbox: double-umount + remount /proc to prevent exfiltration when agent peels off --mount-proc overlay. webhooks: filter check_suite.pull_requests to same-repo PRs only (excludes cross-fork sync PRs) and skip merged/closed PRs. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Colin McDonnell
|
a7bd746f21 |
Restructure dash (#372)
* Restructure dash * WIP * WIP * refactor trigger UI: extract PR summary card, add mentions section, rename labels Co-authored-by: Cursor <cursoragent@cursor.com> * clean up console UI: remove info icons from section descriptions, rename mentions trigger Co-authored-by: Cursor <cursoragent@cursor.com> * fix review feedback: layout, terminology, form scope - extract console sidebar sections to module-level constant - align three-column layout breakpoints to xl (match sidebar visibility) - fix mixed shell/bash terminology in beta page - scope FormProvider to trigger sections only, restore autoComplete="off" Co-authored-by: Cursor <cursoragent@cursor.com> * Bump --------- Co-authored-by: Cursor <cursoragent@cursor.com> |