* fix: replace domain-specific exit handler with generic signal handler registry
Rewrite exitHandler.ts as a generic, domain-agnostic exit signal module
that exports onExitSignal(handler) returning a dispose function.
- subprocess.ts now registers via onExitSignal instead of direct
process.on(SIGINT/SIGTERM) calls
- resolveTokens registers a signal handler that captures tokens by
closure, fixing the race condition where the exit handler would
read the wrong token after disposal
- Remove setupExitHandler and runCleanup — domain cleanup is handled
by post.ts + await using
Co-authored-by: Cursor <cursoragent@cursor.com>
* tweaks
* simplify handler installation
* extract to util
* fix race in dispose
* wrap dispose body in try/finally to ensure disposingRef always settles
---------
Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* Stop using command-based logs for warnings and errors
Co-authored-by: Cursor <cursoragent@cursor.com>
* revert
* tweak
* de-noise
* Remove redundant ts() timestamp prefix from log calls
* Restore timestamped logging and refine debug output routing.
Bring back timestamp prefixes for standard logs and make log.debug emit via core.debug when runner debug is enabled, while still surfacing debug lines for --debug runs.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Log the instructions box immediately after instruction resolution in main, and standardize agent permission summaries to debug-level "disallowed built-ins" output to reduce confusion with pullfrog MCP tools.
Co-authored-by: Cursor <cursoragent@cursor.com>
Increment @pullfrog/pullfrog from 0.0.163 to 0.0.164 and include current beta/docs/waitlist script changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* add workflows permission to git token and waitlist improvements
- add `workflows` to `InstallationTokenPermissions` type in both action and API token routes
- include `workflows: write` in the git token so agents can push workflow file changes
- add `githubFollowers` field to WaitlistSignup schema with migration
- add script to populate waitlist followers from GitHub API
- add frog-green-square-border logo asset
Co-authored-by: Cursor <cursoragent@cursor.com>
* improve CI, agent logging, token permissions, and delegation guardrails
- add format check and build step to root CI job
- standardize agent model/effort log lines across all agents
- fix GitHub App permissions types to match OpenAPI schema (workflows is write-only)
- improve delegation error message to prevent subagent recursion
- demote noisy OpenCode stderr to debug level
- add subagent delegation rules to resolved instructions
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix graphql partial error handling, update delegation message, add workflow_run fixtures
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove module-level env var throws that break CI build
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix logging bug and type hole from PR review
- use batch-local notFound counter so per-batch log doesn't undercount
- add workflows to WorkflowTokenPermissions so wire type matches what action sends
Co-authored-by: Cursor <cursoragent@cursor.com>
* lazy-init appOctokit to fix next build without env vars
Co-authored-by: Cursor <cursoragent@cursor.com>
* drop pnpm build from CI test workflow
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix delegate-effort test regex to match actual log format, disable fail-fast for agnostic tests
the test was matching `running \w+ with effort=auto` but the actual log
line from shared.ts is `» effort: auto`. also temporarily set
fail-fast: false on action-agnostic so all failures surface at once.
Co-authored-by: Cursor <cursoragent@cursor.com>
* disable fail-fast in action workflow too, relax ci.test.ts to match
both workflow files now use fail-fast: false for agnostic tests so all
matrix jobs run to completion. the ci consistency test now checks that
the two workflows agree rather than requiring true.
Co-authored-by: Cursor <cursoragent@cursor.com>
* restore fail-fast: true now that all agnostic tests pass
Co-authored-by: Cursor <cursoragent@cursor.com>
* skip agent tests in CI when agent harness file didn't change
adds action/test/changed-agents.sh which reads the PR diff (via
dorny/paths-filter) and outputs only agents whose harness file was
modified. the action-agents matrix now uses this dynamic list instead
of a hardcoded array, so e.g. a PR touching only cursor.ts runs 6
jobs instead of 30.
Co-authored-by: Cursor <cursoragent@cursor.com>
* update ci.test.ts to validate dynamic agent matrix
the test now checks that the matrix references the changes job output
and that changed-agents.sh correctly discovers all agents.
Co-authored-by: Cursor <cursoragent@cursor.com>
* parallelize action jobs and use claude canary fallback for shared changes
runs action-agents in parallel with action-agnostic after root/changes, and updates changed-agents logic so shared or non-harness action runtime changes run only claude while harness-specific edits run only those changed agents.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* console UI improvements and cleanup
- add verify workflow button and API endpoint for manual installation check
- move env var check into PromptBox as blocking overlay (hoisted to RepoConsole)
- extract FlagsCheatSheet modal, replace verbose flag hints everywhere
- add info popovers for repo setup / post-checkout script descriptions
- remove unused prAutoFixCiFailures schema fields and migration
- default mentionAllowNonCollaborator to disabled for safety on public repos
- update docs for triggers and getting started
Co-authored-by: Cursor <cursoragent@cursor.com>
* add diagnostic logging for push_branch bug investigation
temporary [push-debug] logs to trace why getPushDestination falls back
to origin/<localBranch> instead of using the correct remote branch name
for same-repo PRs.
Co-authored-by: Cursor <cursoragent@cursor.com>
* add git config diagnostic to verify original bug cause
Co-authored-by: Cursor <cursoragent@cursor.com>
* temporarily disable StoredPushDest to test git config path
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove diagnostic logging for push_branch investigation
verified that StoredPushDest fix works correctly on preview repo.
both the stored dest path and the git config fallback resolve to the
correct remote branch in the GitHub Actions environment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix formatting in AgentSettings and TriggersSettings
Co-authored-by: Cursor <cursoragent@cursor.com>
* pass derived env var state to PromptBox instead of raw secrets data
eliminates duplicated derivation logic between RepoConsole and PromptBox
by passing envVarMissing, envVarChecking, and agentKeyNames as props.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: prevent duplicate comment after PR review deletes progress comment
progressCommentId now uses three states: undefined (no comment yet),
number (active), null (deliberately deleted). After create_pull_request_review
deletes the progress comment, subsequent report_progress calls skip instead
of creating a new comment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* effort descriptions, test ordering, husky, docs images, typo fix
- rewrote delegation effort level descriptions to per-level breakdown
- action-agents now waits for action-agnostic; action-agnostic waits for root
- added husky + lint-staged (biome check --write on staged files)
- updated triggers docs images and triggers.mdx content
- fixed "figured" → "figures" typo on landing page
- updated pnpm-lock.yaml
Co-authored-by: Cursor <cursoragent@cursor.com>
* Commit
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
reads (file_read, list_directory) now allow paths within the repo OR
PULLFROG_TEMP_DIR. this fixes the bug where agents with full permissions
couldn't read PR diffs, CI logs, review threads, or background bash
output because those files live under /tmp/pullfrog-xxx/.
writes (file_write, file_edit, file_delete) now only enforce repo-scoping
when bash !== "enabled". when bash=enabled the agent can write anywhere
via native bash, so restricting file_write was security theater. .git/
stays blocked in all modes as defense-in-depth.
replaced the conflated resolveAndValidatePath/validateWritePath helpers
with separate resolveReadPath and resolveWritePath functions that cleanly
separate path resolution from permission enforcement.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test preview bypass 2
Co-authored-by: Cursor <cursoragent@cursor.com>
* add apiFetch wrapper with Vercel bypass via query param + header
the template workflow was missing VERCEL_AUTOMATION_BYPASS_SECRET,
so all action API calls to preview deployments hit Vercel's
deployment protection without bypass. this also consolidates the
bypass logic into a single fetch wrapper that applies the secret
as both a query parameter (matching server-side forwarding) and
a header for belt-and-suspenders reliability.
Co-authored-by: Cursor <cursoragent@cursor.com>
* security hardening for Vercel bypass
- redact bypass token from webhook forwarder logs and response body
- remove dead x-preview-api-forward header
- refactor getAllSecrets() to use SENSITIVE_PATTERNS instead of hardcoded list
- enforce https:// on API_URL (localhost exempt for local dev)
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
getPushDestination used git's @{push} which under push.default=simple
resolves using the local branch name as the remote branch name. since
checkout_pr uses pr-N as the local name, this resolved to origin/pr-N
instead of the actual PR branch (e.g. origin/pullfrog/feature-branch).
this caused two failure modes:
- agent passes remote branch name to push_branch → "src refspec does
not match any" because no local branch has that name
- agent calls push_branch with no args → silently pushes to a new
remote branch pr-N instead of updating the PR branch
fix: read branch.X.pushRemote and branch.X.merge from git config
directly (the exact config checkout_pr already writes) instead of
relying on @{push}. also rename head → localBranch + remoteBranch
in CheckoutPrResult to make the distinction explicit.
Co-authored-by: Cursor <cursoragent@cursor.com>
action API calls to preview deployments were getting 401'd by Vercel's
deployment protection. add x-vercel-protection-bypass header to the 3
server-to-server fetch sites when VERCEL_AUTOMATION_BYPASS_SECRET is set.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Add `comment_type: "issue"` to `IssueCommentCreatedEvent` interface and
dispatch sites so agents can distinguish issue comments from PR review
comments
- Add dedicated "Progress reporting" section to system prompt making
`report_progress` the mandatory tool for sharing results
- Update `reply_to_review_comment` description to clarify it only works
for inline review comments on PR diffs, not issue comments
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* Use a stable `NEXT_PUBLIC_VERCEL_BRANCH_URL` for short links
* Update JSDoc reference to NEXT_PUBLIC_VERCEL_BRANCH_URL
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* pin all CLI installations to explicit versions and use pro models by default
- codex: pin @openai/codex to 0.101.0 (was "latest")
- opencode: pin opencode-ai to 1.1.56 (was "latest")
- gemini: pin gemini-cli to v0.28.2 via new tag param on installFromGithub
- cursor: pin to 2026.01.28-fd13201 via direct tarball download (replaces curl install script)
- add installFromDirectTarball to install.ts for versioned tarball URLs
- gemini auto effort now uses pro-preview instead of flash-preview
- test runner model overrides updated to use pro-preview consistently
Co-authored-by: Cursor <cursoragent@cursor.com>
* increase activity timeout
* fix delegation timeout
* fix delegate timeouts
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* pin all CLI installations to explicit versions and use pro models by default
- codex: pin @openai/codex to 0.101.0 (was "latest")
- opencode: pin opencode-ai to 1.1.56 (was "latest")
- gemini: pin gemini-cli to v0.28.2 via new tag param on installFromGithub
- cursor: pin to 2026.01.28-fd13201 via direct tarball download (replaces curl install script)
- add installFromDirectTarball to install.ts for versioned tarball URLs
- gemini auto effort now uses pro-preview instead of flash-preview
- test runner model overrides updated to use pro-preview consistently
Co-authored-by: Cursor <cursoragent@cursor.com>
* increase activity timeout
* fix delegation timeout
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Update Build, AddressReviews, and Prompt modes to ensure tests are run
BEFORE committing and pushing code. This prevents redundant workflow
triggers when tests fail and need fixes.
Previously, the Build and Prompt modes would:
1. Make code changes
2. Commit and push
3. Test (oops, too late!)
Now all modes follow the correct order:
1. Make code changes
2. Test (if tests fail, fix and repeat)
3. Commit and push
This addresses the inefficiency observed in #268 where the agent pushed
code before verifying it worked, then had to fix and push again.
Closes#273
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* Replace Date.now() with performance.now() for duration measurements
- Import performance from node:perf_hooks in all affected files
- Update Timer and ThinkingTimer classes to use performance.now()
- Update activity tracking (markActivity, getIdleMs) to use performance.now()
- Update cache duration measurements to use performance.now()
- Update agent execution timing (cursor, opencode) to use performance.now()
- Update subprocess execution timing to use performance.now()
- Update API performance monitoring to use performance.now()
- Update prep phase timing to use performance.now()
- Update timer.test.ts to mock performance.now() instead of Date.now()
Benefits:
- Monotonic clock immune to system clock adjustments
- Higher precision (microsecond vs millisecond resolution)
- Purpose-built for performance measurement
Fixes#245
* fix lint.
* Round float durations to integers in logging
Preserve original behavior by rounding performance.now() float values
to integers when displaying/logging millisecond durations.
* fix lint.
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Robin Tail <robin_tail@me.com>
* upgrade Claude to Opus 4.6 with --effort max for --max mode
- mini: haiku → sonnet
- auto: opusplan → opus (Opus 4.6)
- max: opus → opus + --effort max (Opus 4.6 max effort)
- bump @anthropic-ai/claude-agent-sdk 0.2.7 → 0.2.39
Co-authored-by: Cursor <cursoragent@cursor.com>
* update action lockfile for claude-agent-sdk 0.2.39
Co-authored-by: Cursor <cursoragent@cursor.com>
* add tool_use_summary handler for SDK 0.2.39
Co-authored-by: Cursor <cursoragent@cursor.com>
* integrate gpt-5.3-codex with runtime model availability detection
checks GET /v1/models at agent start to determine if gpt-5.3-codex is
available for the API key, falling back to gpt-5.2-codex when it isn't.
model resolution runs concurrently with CLI install for zero added latency.
also bumps @openai/codex-sdk from 0.80.0 to 0.98.0.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* make waitlist code field required
all existing rows have been backfilled with unique codes via the consolidation script.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix lint errors
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(agents): add thinking time logging between tool calls
Adds a ThinkingTimer utility that tracks the gap between tool results and
the next tool call. When the gap exceeds 3 seconds, it logs the duration
with a stopwatch emoji (⏱️ 4.2s).
Uses performance.now() for high-resolution timing and Intl.NumberFormat
for rendering duration in seconds with optional fraction digits.
Integrated across all 5 agents: Claude, Codex, Cursor, Gemini, OpenCode.
Closes#127
* fix: adjusting tests for mocking performance.now.
* fix: reducing diff for claude.
* fix: rm unused args for claude.
* rm unused args for codex.
* fix: rm unused args for gemini.
* fix: rm unused args for opencode.
* mv THINKING_THRESHOLD.
* rev: I decided to pospone node:perf_hooks integration since it requires more comprehensive refactoring.
* fix: using Intl unit formatting.
* tests for ThinkingTimer.
* fix: narrow unit.
* fix: making durationFormatter a class instance property since using one agent per run.
* fix: inverting condition in markToolCall.
* thinking timer improvements and fix actions/checkout v6 auth
- thinking timer: use » chevron and "thought for X seconds" format
- thinking timer: add debug timestamps for sanity checking
- demote PID namespace isolation logs to debug
- remove redundant "setting up git authentication" log
- fix duplicate Authorization header with actions/checkout v6: clean up
includeIf credential entries that v6 persists via external config files
Co-authored-by: Cursor <cursoragent@cursor.com>
* standardize tool call log prefix to » double chevron
Co-authored-by: Cursor <cursoragent@cursor.com>
* update timer tests for new thinking log format
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Robin Tail <robin_tail@me.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(action): Do cleanup when workflow failed or cancelled early.
* fix: avoid naming collision with get-installation-token state.
* tmp: add more logging for debugging purposes.
* fix: rm wasUpdated state.
* FIX: changing approach, separate entrypoint, using db to handle cases when main() never ran.
* fix(cleanup): revert changes to exitHandler.
* FIX: using event payload for issue and comment retrieval instead of DB.
* FIX: use installation token.
* feat(docs): wiki article explaining how it works.
* fix(docs): shortening.
* fix(docs): shortening.
* fix: no console.
* todo: DNRY for findPullfrogComment.
* FIX(DNRY): upgrading octokit/rest and reusing findInitialComment() from triggerWorkflow.ts.
* Revert "FIX(DNRY): upgrading octokit/rest and reusing findInitialComment() from triggerWorkflow.ts."
This reverts commit 7dd239ba0986c5b0aeacb6ddc9f2deddb83aee82.
* fix: rm todo.
* fix(DNRY): shortening early exit logging statements.
* FIX(API): Avoid extra call for comment body.
* feat(DNRY): extracting and reusing buildWorkflowErrorMessage() from exitHandler.ts.
* fix(DNRY): extracting more similarities into buildErrorCommentBody.
* feat: Add conditional reason check.
* fix(merge): replacing resolveInstallationToken with getJobToken.
* fix(debug): using higher severity.
* fix: Adjusting the implementation of getIsCancelled to use job status instead of workflow.
* fix: Take steps conclusion into account when job is in progress.
* fix: generic log msg.
* fix: jsdoc.
* fix(docs): Updating the wiki article according to recent changes.
* fix(post): Handling the case when current job runs within matrix.
* fix: finding the most recent comment that is ours ANS stuck.
* feat(opt): using progressCommentId from object-based prompt when present.
* fix(docs): Shortening the documentation 3 times down.
* FIX: Only using the prompt.progressCommentId but with validation that it is stuck.
Add `lint`, `lint:fix`, `format`, and `format:fix` package.json scripts
backed by biome. Add AGENTS.md rule for agents to run them after changes.
Fix all existing lint and format violations across the codebase.
Co-authored-by: Cursor <cursoragent@cursor.com>
* flatten lifecycle hooks into RepoSettings string fields
replace the separate LifecycleHook model with setupScript and
postCheckoutScript string fields directly on RepoSettings. move the UI
into the Agent settings section alongside environment variables and
custom instructions. delete the standalone lifecycle-hooks API route,
component, and schema since the existing settings PATCH endpoint
handles the new fields automatically.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: pass env to lifecycle hook spawn so scripts can use package managers
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Scope installation token permissions in restricted mode
In restricted/disabled bash mode, the installation token is now scoped
to match the workflow's permissions block. This preserves fork push
capability while limiting what the agent can do with the token.
- Read workflow permissions from pullfrog.yml at runtime
- Pass permissions to API when acquiring installation token
- Clear OIDC env vars in restricted mode to prevent token minting
- Simplify setupGit by moving token resolution to main.ts
* Address review feedback: fail closed with default permissions
- Add restrictive default permissions (contents:read, pull_requests:read,
issues:read) as fallback when workflow permissions can't be read
- Add support for job-level permissions via GITHUB_JOB env var
- Fix misleading comment about token resolution in restricted mode
- Add documentation about fork PR checkout behavior
* Simplify to separate git/MCP tokens without workflow permission scoping
- gitToken: minimal contents:write only (assumed exfiltratable)
- mcpToken: full installation token (not exfiltratable via MCP tools)
- Remove workflowPermissions.ts - security-conscious users can pass
their own token via GH_TOKEN or inputs.token
- Add type-safe InstallationTokenPermissions to github.ts and API route
* Rename `write` permission to `push` and remove vestigial tool blocking
The `write` permission was previously used to block local file write tools
in agents. This was security theater since bash can write files anyway.
Now `push` only controls the git token scope:
- push: enabled → contents:write (can push commits)
- push: disabled → contents:read (read-only, can't push)
Changes:
- Rename `write` to `push` in action.yml, Prisma schema, and all TS types
- Remove vestigial write tool blocking from all agents (claude, cursor,
gemini, opencode, codex)
- Add data-preserving Prisma migration using RENAME COLUMN
- Update UI: "Write files" → "Git push" with updated description
* add PID namespace isolation for bash sandbox
when running in CI, attempts to use unshare --pid to create a new PID
namespace for bash subprocesses. this prevents the /proc/$PPID/environ
attack where a malicious command could read secrets from the parent
process's environment.
the protection works by:
1. creating a new PID namespace (subprocess becomes PID 1)
2. mounting fresh /proc showing only sandbox PIDs
3. parent PIDs become invisible (PPID = 0, /proc/0 doesn't exist)
combined with filterEnv(), this provides complete protection against
/proc-based secret theft. falls back gracefully if namespaces aren't
available.
includes test script to verify the protection works.
* add PID namespace test to CI workflow
tests whether unshare --pid works on GHA runners out of the box,
and if not, whether enabling via sysctl helps. also runs the
pidNamespace.ts test to verify the full protection.
* fix pnpm setup and add procIsolation agent test
- fix pnpm/action-setup by specifying package_json_file path
- add procIsolation crossagent test that has agent attempt to
read secrets via /proc/$PPID/environ
- add procIsolation to CI test matrix
* add pid-namespace test job to main workflow
this job tests unshare --pid capabilities on GHA runners and runs
the pidNamespace.ts adhoc test to verify /proc isolation works
* test bubblewrap's sysctl approach for enabling namespaces
- write to /etc/sysctl.d/99-userns.conf and run sysctl --system
- try aa-complain on unshare binary
- more detailed diagnostics
* fix pidNamespace test and add sudo-unshare fallback for GHA
- fix reference error in pidNamespace.ts (renamed function but didn't update calls)
- add sudo-unshare as fallback method for GHA runners where unprivileged
namespaces are blocked but sudo is available
- update bash.ts to detect and use sudo unshare when unprivileged fails
* consolidate security docs and document PID namespace isolation
- update security.md with current implementation details
- document sudo unshare fallback for GHA runners
- add testing instructions for local Docker and CI
- add "Further Exploration" section with Landlock and path validation ideas
- delete bash-sandbox.md and landlock.md (consolidated into security.md)
* move procIsolation test to adhoc folder
the procIsolation test requires PID namespace capabilities that aren't
available in the Docker test environment. moved to adhoc/ so it's excluded
from default test runs and can be run explicitly when needed (e.g. via
the pid-namespace CI job or locally with --privileged docker).
* fix Docker test environment for PID namespace isolation
- add CI and GITHUB_ACTIONS to testEnvAllowList so sandbox detection runs
- add --privileged to Docker run for PID namespace support (unshare)
this fixes the test environment to properly test the sandbox. in production,
the action runs directly on GHA runner where sudo unshare works.
* fix getJobToken() to work in test environment
add fallback to GH_TOKEN and GITHUB_TOKEN when INPUT_TOKEN is not set.
this allows tests to run without requiring workflow-level token input.
the token resolution order is:
1. INPUT_TOKEN (from workflow `with: token:`)
2. GH_TOKEN (external token override)
3. GITHUB_TOKEN (pre-acquired in tests or from GHA env)
* security: filter secrets from all subprocess environments
- extract filterEnv() to shared utils/secrets.ts
- make $() utility filter secrets by default (git, npm, etc. don't need them)
- disable git hooks via core.hooksPath to prevent hook-based exfiltration
- git auth uses token embedded in URL, not env vars
this prevents malicious git hooks, npm postinstall scripts, and other
code execution vectors from exfiltrating GITHUB_TOKEN and API keys.
* docs: clarify defense-in-depth security model
update security.md to explain why BOTH layers are required:
- filterEnv(): cleans child's own /proc/self/environ
- PID namespace: hides parent's /proc entries
PID namespace alone isn't sufficient - with --mount-proc, the child
becomes PID 1, so /proc/1/environ is the child's OWN environment.
without filterEnv(), secrets would still be accessible there.
* add procSandbox crossagent test for PID namespace security
- add crossagent/procSandbox.ts: security test that instructs agent to try
various /proc attack vectors and validates sandbox blocks them
- update wiki/security.md: document PID namespace isolation details, add
verification commands, explain why sudo inside sandbox doesn't break security
- update docker.ts: use node:24 with sudo for GHA-like test environment
- update instructions.ts: allow disabling security messaging for pentests
- clean up adhoc test files (procIsolation.ts, securityAudit.ts)
the procSandbox test sets SANDBOX_TEST_TOKEN (matches *_TOKEN filter), gives
the agent explicit hints about attack vectors (/proc, sudo, nsenter, etc.),
disables prompt-based mitigations, and validates the secret is never leaked.
* move procSandbox test to agnostic/ (runs with one agent)
* WIP
* docs: add agent testing guide (pnpm play, Docker, pentesting)
* docs: add CI details to agent testing guide
* docs: add interesting findings and gotchas from pentesting
* improve test fidelity: auto-set CI=true, verify sandbox active
- docker.ts: always set CI=true in container so sandbox activates
- docker.ts: skip sudo/user setup if already done (faster reruns)
- bash.ts: export getSandboxMethod() for diagnostics
- bash.ts: add debug log when sandbox disabled
- procSandbox.ts: add sandbox_was_active check to catch vacuous passes
the CI=true change is critical: without it, PID namespace isolation
is skipped and security tests pass without actually testing anything.
* docs: update agent-testing.md with CI=true auto-set note
* docs: clarify log format is agent-specific
* fix git auth, simplify MCP tools, add adversarial tests
- fix gitWithAuth to use Basic auth format (Bearer doesn't work with git's http.extraheader)
- fix token.ts: push:restricted now correctly gets contents:write
- fix github.ts: pass permissions when acquiring installation tokens locally
- remove commit_files and create_branch MCP tools (redundant, don't require credentials)
- remove containsSecrets() - trivially bypassable, not sound security
- add agnostic adversarial tests: pushDisabled, pushRestricted, tokenExfil
- update instructions.ts to clearly list available git MCP tools
- add wiki/git.md documenting credential isolation and push permission tiers
- update wiki/docker.md with custom image considerations
- update wiki/agent-testing.md with adversarial testing patterns
* fix type errors after rebase
- change ResolveTokensParams.push from ToolPermission to PushPermission
- use tags: ["agnostic"] instead of agnostic: true in test files
* fix cleanup permission error in sandbox tests
when sandbox isolation is enabled (CI=true), files created by the unshare
subprocess may have different ownership, causing rmSync to fail with EACCES.
this error in the finally block was overriding the test's success result.
fix: wrap cleanup in try-catch and fall back to sudo rm if rmSync fails.
* Add adhoc
* Handle git config/remote bypasses
* add git hooks protection and simplify ToolState
- disable git hooks in restricted mode via -c core.hooksPath=/dev/null
- add gitHooks adversarial test to verify hook protection works
- unify prNumber/issueNumber into single issueNumber field
- add pushUrl to ToolState for push validation
- add generateTestMarker() for simpler single-agent test markers
- export SENSITIVE_PATTERNS and isSensitiveEnvName from secrets.ts
- remove redundant pidNamespace.ts (duplicated by procSandbox.ts)
- update documentation
* harden $git() auth: subcommand whitelist, binary tamper detection
- rename gitWithAuth() to $git() with explicit subcommand first arg
- restrict to "fetch" | "push" at type level (filters don't run for these)
- resolve git binary path at startup via resolveGit(), sha256 fingerprint
- verify hash before each $git() call to detect binary replacement
- rename disableHooks to restricted for cleaner semantics
- document filter exfiltration attack and empirical verification in wiki
* remove redundant pid-namespace CI job
the PID namespace isolation testing is now handled by
action/test/agnostic/procSandbox.ts via pnpm runtest agnostic
* fix push_branch for new branches and improve token leak detection
- getPushDestination now falls back to origin/<branch> when @{push}
is not configured (happens for new branches created locally)
- gitPerms validator now checks for actual token patterns instead
of matching "x-access-token" string in test instructions
* use kebab-case for test names
* simplify shell env API: "restricted" | "inherit" | object
replace passFullEnv boolean with cleaner env option that accepts:
- "restricted" (default): filterEnv() to prevent secret leakage
- "inherit": full process.env
- object: custom env merged with restricted base
* share EnvMode and resolveEnv between shell.ts and bash.ts
move shared env resolution logic to secrets.ts
* add env option to bash tool (default: restricted)
* delete agent-testing.md (renamed to adversarial.md)
* Add checkout tests
* reframe githooks test prompt to avoid claude safety refusal
claude was refusing to execute the test because the prompt used words
like "malicious" and "security testing". reframed as a debugging task
with innocuous env var name (TESTING_DEBUG_TAG) per adversarial.md guidance.
Co-authored-by: Cursor <cursoragent@cursor.com>
* clean up verbose token acquisition logs
move logging responsibility to call sites which have better context
(git token vs MCP token). remove redundant intermediate OIDC logs
and unused "(permission-scoped)" suffix.
Co-authored-by: Cursor <cursoragent@cursor.com>
* isolate agnostic tests with matrix strategy, fix .pullfrog-env secret leak
- split action-agnostic into per-test matrix jobs for isolated logs and filesystems
- only write explicitly opted-in env vars to .pullfrog-env via fileAgentEnv
(fixes token-exfil test where claude found SANDBOX_TEST_TOKEN on disk)
- mcpmerge test opts in via fileAgentEnv for cursor's repo-level MCP fallback
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove env parameter from bash tool to prevent agents bypassing filterEnv
the bash tool exposed an `env` parameter accepting "restricted" | "inherit"
which allowed agents to pass env: "inherit" and see all secrets including
SANDBOX_TEST_TOKEN, bypassing the restricted environment filtering entirely.
env mode is now determined internally (always restricted).
Co-authored-by: Cursor <cursoragent@cursor.com>
* use pullfrog/test-repo for push tests to stop polluting main repo
push tests were creating branches and tags on pullfrog/app directly.
now all push tests (push-restricted, push-disabled, push-enabled,
git-permissions) target pullfrog/test-repo instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
* use pullfrog/test-repo for all tests, not just push tests
no test should clone or operate on pullfrog/app directly.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix token scoping for test-repo and bash timeout defaults
- acquireTokenViaOIDC now includes GITHUB_REPOSITORY repo in token
scope so push tests work against pullfrog/test-repo
- bash tool default timeout: 120s -> 30s, cap: 600s -> 120s
- activity timeout: 30s -> 60s
- prevents hung bash commands (e.g. find /) from killing the agent
via activity timeout
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
David Blass