* fix false "without reporting progress" error + live todo tracking
clean up orphaned progress comments when review is skipped or only
set_output is used, preventing the false positive in handleAgentResult.
parse todowrite events from OpenCode's NDJSON stream and render a
live markdown checklist in the PR progress comment (2s debounce).
agent's explicit report_progress always takes priority.
Made-with: Cursor
* fix contradictory review/progress prompting
align Review and IncrementalReview mode prompts with their guidance —
mode prompts said "always submit" while guidance said "skip if clean."
remove the empty-approval submission that was silently dropped by the
tool. make progress comment lifecycle explicit: created on first call,
updated in place, removed after review submission.
Made-with: Cursor
* centralize todo tracking into shared TodoTracker module
extract inline todo tracking logic (~95 lines) from opentoad.ts into
action/utils/todoTracking.ts. the tracker is created once in main.ts
and passed to agents via AgentRunContext.todoTracker, making it
agent-agnostic and reusable for future agent implementations.
Made-with: Cursor
* fix todoTracker optional type to match file convention
add | undefined to todoTracker in AgentRunContext, matching every
other optional property in the same interface.
Made-with: Cursor
* instruct agents to always maintain a task list for live progress
system prompt now tells agents to create an internal task list at
the start of every run. the tracker renders it to the progress
comment automatically. report_progress is reserved for final
results only — no more intermediate "Checking..." messages that
cancel the tracker and leave stale text on the comment.
Made-with: Cursor
* require report_progress summary at end of every run
agents must always call report_progress with a final summary —
the completed task list should never be the end state of the
progress comment. updated all review mode prompts to call
report_progress after submitting (or not submitting) a review.
Made-with: Cursor
* keep progress comment after review with final summary
stop deleting the progress comment after review submission —
the agent now always calls report_progress with a summary at
the end, and that summary should persist as a record of what
was done.
Made-with: Cursor
* harden stranded progress comment cleanup
- main.ts: detect when tracker was last writer (agent never called
report_progress) and delete the stranded checklist instead of
leaving it as the final comment state
- postCleanup.ts: expand stuck-comment detection to also catch
stranded todo checklists (regex match for checklist patterns)
when the process is killed before normal cleanup runs
- modes.ts + selectMode.ts: add report_progress step to Summarize
and SummaryUpdate modes (only modes that were missing it)
Made-with: Cursor
* fix stale comments, typo, and build mode redundancy
- comment.ts: update deleteProgressComment docstring and inline comment
to reflect current usage (stranded-comment cleanup, not post-review)
- modes.ts: merge duplicate report_progress steps (8 + 10) into single
step 9, fix "optimizatfixons" typo
- wiki/post-cleanup.md: document checklist detection regex
Made-with: Cursor
* collapsible completed todos in final progress, hide set_output outside standalone mode
- add renderCollapsible() to TodoTracker, append completed task list as
<details> section when agent calls report_progress
- cancel tracker after agent's final report_progress so it doesn't
overwrite with raw checklist
- conditionally register SetOutputTool only in standalone mode or when
output_schema is provided
- remove unconditional set_output instruction from orchestrator task section
- update Summarize/SummaryUpdate mode guidance to not reference set_output
Made-with: Cursor
* show completion count in collapsible task list summary
Made-with: Cursor
* only count completed (not cancelled) in collapsible task list summary
Made-with: Cursor
* reinforce concise summary prompting across system prompt, modes, and tool description
Made-with: Cursor
* address review feedback: wasUpdated bypass, tracker false-positive, race condition
- remove wasUpdated=true from cleanup paths so handleAgentResult correctly
detects genuinely silent runs
- add hadProgressComment to ToolState as immutable snapshot for the safety check
- use todoTracker.hasPublished instead of enabled for stranded-comment cleanup
- serialize onUpdate calls via inflightPromise chain with post-cancel guard
- add settled() to wait for in-flight updates before writing final summary
Made-with: Cursor
* address round-2 review: hasPublished after success, finalSummaryWritten flag
- set hasPublished only after onUpdate resolves (not before) so failed
writes are not counted as published
- add finalSummaryWritten flag to ToolState, set after successful
non-plan reportProgress; decouple cleanup detection from
todoTracker.enabled so it survives API failures where cancel() ran
but the write didn't succeed
Made-with: Cursor
* add repo learnings feature with edit history
introduces a new "Learnings" section in the repo console where agents can
persist operational knowledge (setup steps, test commands, conventions) at
the end of runs via an MCP tool. users can also edit learnings manually.
- add `learnings` field to Repo model and `LearningsRevision` audit table
- add `update_learnings` MCP tool for agents to persist repo knowledge
- integrate learnings into prompt assembly as REPO LEARNINGS section
- add learnings step to mode guidance (Build, AddressReviews, Plan, Fix, Task)
- add PATCH /api/repo/[owner]/[repo]/learnings endpoint (JWT auth)
- add GET /api/repo/[owner]/[repo]/learnings/history endpoint (Clerk auth)
- add LearningsSection component with textarea, save-on-blur, and history modal
- record revision history with actor tracking (agent vs user) and pruning (50 max)
- gate UI behind owner === "pullfrog" for internal dogfooding
Made-with: Cursor
* fix prisma enum import path for LearningsActor
Made-with: Cursor
* simplify learnings schema: remove LearningsActor enum, store model name directly
the actor/actorName split was unnecessary — learnings are only written by
agents so the revision table just needs a model column. removes all user
editing concepts from schema, API, and frontend.
Made-with: Cursor
* fix migration: add separate migration instead of rewriting existing one
restores original learnings_revisions migration and adds a new migration
that drops actor/actorName columns, backfills model from actorName, and
drops the LearningsActor enum.
Made-with: Cursor
* polish learnings feature: rename to Repo Intelligence, fix atomicity, fix review skip
- rename user-facing "learnings" to "Repo Intelligence" (UI, prompt section, wiki, sidebar)
- simplify description to "Automatically discovered by the agent across runs."
- wrap repo.update + revision create in $transaction for atomicity
- refactor recordLearningsRevision to pruneLearningsRevisions (prune-only)
- fix empty review skip: don't block APPROVE reviews with no body
- fix broken docs anchor: #free-options → #free-models
- update agent guidance to require flat bullet list format with pruning
- add accessibility: aria-expanded, sr-only loading, output element
- add chevron rotation, stale data clear on modal close, max-h scroll
- trim + length-limit model field, remove type cast, restore pre-existing comment
- update wiki prompt examples with actual bullet-formatted content
- update model test snapshot
Made-with: Cursor
* fix stale free model name in docs, rename utility file to match export
- docs/keys.mdx: MiMo V2 Flash → MiMo V2 Pro (matches model code change)
- rename recordLearningsRevision.ts → pruneLearningsRevisions.ts
Made-with: Cursor
* Add skill, .neon
* polish learnings UI and remove verbose log
- learnings code block: read-only appearance with muted text, copy button, rounded corners
- history modal: full-width rows with cursor-pointer, chevron moved to right, no preview text
- drop noisy update_learnings log line
Made-with: Cursor
* inject learningsStep into all modes, drop seed script, soften revision styling
Made-with: Cursor
* Drop seed
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* agent & model refactor with ASKPASS git auth, UI restructure, clerk v7
Made-with: Cursor
* fix stale agent/effort refs, add tests for askpass + model resolution
- reviewCleanup.ts: payload.agent -> payload.model, remove effort
- selectMode.ts PlanEdit: remove delegation/subagent/effort references
- pullfrog.yml.ts: update env vars (drop GOOGLE_API_KEY/CURSOR_API_KEY,
add GOOGLE_GENERATIVE_AI_API_KEY/XAI_API_KEY/MOONSHOT_API_KEY/OPENCODE_API_KEY)
- FlagsSettings/RepoInstructionsSection: remove stale effort/timeout copy
- new: gitAuthServer.test.ts (10 tests — lifecycle, token delivery, tamper detection, script gen)
- new: agent.test.ts (4 tests — default opentoad, AGENT_OVERRIDE, invalid override)
- new: models.test.ts (19 tests — parseModel, resolution, registry invariants)
- update models.dev snapshot
Made-with: Cursor
* fix changed-agents.sh to filter legacy agent files from CI matrix
legacy agent files (claude.ts, codex.ts, etc.) are @ts-nocheck and not
exported from index.ts. changed-agents.sh now reads index.ts imports to
build the active agent set and treats changes to inactive files as
non-agent changes (opentoad canary only).
Made-with: Cursor
* remove MCP file tools, old agent harnesses, and obsolete security tests
ASKPASS-based git auth makes the old MCP file tool security layer unnecessary:
- token never in subprocess env, so symlink/gitattributes/hook attacks can't exfiltrate it
- agents now use native file tools (OpenCode builtin read/edit)
deleted:
- action/mcp/file.ts (file_read, file_write, file_edit, file_delete, list_directory)
- action/mcp/index.ts (dead re-export)
- agent harnesses: claude.ts, codex.ts, cursor.ts, gemini.ts, opencode.ts
- opencode-runner.ts (inlined into opentoad.ts)
- security tests that validated MCP file tool restrictions
- commented-out three-step review flow (~300 lines)
- sanitizeSchema/wrapSchema dead code from mcp/shared.ts
- OPENCODE_MODEL_MINI/MAX env vars (effort-level model overrides removed)
updated test prompts to use generic file ops instead of MCP tool names.
restored pkg-json-scripts + requirements-txt-attack (test --ignore-scripts defense).
Made-with: Cursor
* bump actions/checkout v4 → v6 (node 24)
node 20 actions deprecated june 2, 2026.
Made-with: Cursor
* temporarily disable fail-fast on agnostic tests to debug checkout@v6
Made-with: Cursor
* re-enable fail-fast on agnostic tests
Made-with: Cursor
* fix test token mismatch: mint OIDC tokens scoped to target repo
CI tests override GITHUB_REPOSITORY to pullfrog/test-repo but inherit
the runner's GITHUB_TOKEN (scoped to pullfrog/app), causing 401s on
every run-context fetch. Clear GITHUB_TOKEN in the test subprocess so
ensureGitHubToken() mints a properly scoped token via OIDC.
Also centralizes the default GITHUB_REPOSITORY in runAgentStreaming
instead of repeating it in every test file, and fixes preview-cleanup
to remove workers from all queues (not just name-matching ones).
Made-with: Cursor
* fix ensureGitHubToken to try OIDC when app credentials are absent
ensureGitHubToken only attempted token minting when GITHUB_APP_ID and
GITHUB_PRIVATE_KEY were set. In CI, OIDC is available but app creds
aren't exposed — so the guard prevented minting entirely.
Made-with: Cursor
* dead code cleanup: remove remnants of deleted agents, file tools, effort system
remove unused @anthropic-ai/claude-agent-sdk and @openai/codex-sdk deps,
orphaned file-tool security tests, dead GEMINI_MODEL passthrough, stale
opencode-runner wiki refs, deleted test file references, and MCP file tool
docs. rename docs/effort → docs/models. fix vitest setup: move dotenv to
globalSetup (runs once before forks instead of per-file, 19s → 200ms).
Made-with: Cursor
* address review feedback: remove dead code, update stale references
- remove AGENT_OVERRIDE (only opentoad exists)
- remove shellToolName plumbing (always restricted shell)
- bump action version to 0.0.179
- remove CURSOR_API_KEY from all workflows/configs
- remove OPENCODE_MODEL_MINI/MAX from workflows/docs
- delete wiki/effort.md, rewrite docs/effort.mdx as "Models"
- rewrite wiki/modes.md: orchestrator/subagent → single agent
- simplify flag system: drop builtin flag extraction (debug, effort,
timeout, agent), keep custom flag replacement only
- reserve all legacy flag names to prevent custom flag conflicts
Made-with: Cursor
* regenerate lockfile after removing claude-agent-sdk and codex-sdk
Made-with: Cursor
* fix import ordering, add lockfile check to pre-push hook
Made-with: Cursor
* remove dead debug payload field, stale packageExtensions
Made-with: Cursor
* merge proc-sandbox and token-exfil into a single test
proc-sandbox and token-exfil were duplicative — both tested that
SANDBOX_TEST_TOKEN couldn't be exfiltrated. consolidated into
token-exfil with shell:restricted (which actually exercises filterEnv)
and the /proc attack vector hints from proc-sandbox.
Made-with: Cursor
* fix wiki adversarial.md to match actual tokenExfil validator
Made-with: Cursor
* add mode instructions and restructure dashboard sidebar
- add modeInstructions JSONB field to Repo model for per-mode user instructions
- thread modeInstructions through settings API, run-context API, RepoSettings, ToolContext, and selectMode runtime
- merge user-defined mode instructions with hardcoded orchestrator guidance, with IncrementalReview inheriting from Review
- reduce visible built-in modes from 7 to 4 (Build, Review, Plan, Fix) with editable Instructions textareas
- add TRIGGERS group header to sidebar above Mentions, Pull requests, Issues
- add wiki/modes.md documenting triggers and modes conceptual model
Made-with: Cursor
* fix leaping comment deletion and address review feedback
- wrap post-createReview operations in try/finally so deleteProgressComment
runs even when updateReview or reportReviewNodeId throws
- add parseModeInstructions runtime guard to filter non-string values
from the JSONB field before passing to buildOrchestratorGuidance
- add useEffect sync for localInstructions when props change
- guard onBlur to skip save when instructions haven't changed
- update wiki/modes.md to reflect V2 is implemented (no longer "proposed")
Made-with: Cursor
* harden review cleanup, fix type cast, stabilize mode instructions state
- wrap deleteProgressComment in try/catch inside finally to prevent masking original errors
- replace `as Record<string,string>` cast with runtime parseModeInstructions + useMemo
- fix wiki dual-prompt table to reflect mode.prompt fallback status
Made-with: Cursor
* fix wiki tense and heading ambiguity from PR review
Made-with: Cursor
* fix review "edited" badge by using pending review + submit flow
create review as PENDING first (no event/body), build the footer with
the now-known review ID, then submitReview with the full body. single
atomic publish — no updateReview edit needed.
Made-with: Cursor
* add post-agent follow-up re-review dispatch
After the agent exits, check if PR HEAD moved past the reviewed commit
and dispatch a follow-up re-review. This closes the gap where push
webhooks are suppressed during in-flight reviews.
Made-with: Cursor
* add silent flag to follow-up re-review dispatch
Made-with: Cursor
* restructure dashboard for consistency and clarity
- consolidate tools into single grouped card (was 4 separate cards)
- merge coding + autofix CI into one section
- remove redundant trigger section descriptions
- add bidirectional crosslinks between modes and triggers
- inline instruction links (review/plan/build) into descriptions
- add save status indicators to all sections
- restructure flags with grouped built-in/custom cards
- flatten sidebar (remove dividers and group headers)
- tighten all descriptions
Made-with: Cursor
* update PR screenshots for new dashboard layout
Made-with: Cursor
* extend review context inline instead of dispatching new workflow
when commits are pushed during a review, the agent now handles them
inline: create_pull_request_review detects HEAD movement, returns
instructions to pull and review the incremental diff, and the agent
submits a second review covering only the new changes. this avoids
the cost of spinning up a full new workflow run.
also fixes a bug where reviewedSha was set to the submission HEAD
(current) rather than the checkout HEAD (what was actually reviewed),
which caused commits pushed between checkout and submission to be
silently missed by postReviewCleanup.
the workflow dispatch is kept as a safety net for agent timeout/error.
Made-with: Cursor
* polish dashboard UI: fix debug markers, crosslinks, title consistency, descriptions
- remove all red debug borders/labels and CM component
- remove all inline style={{}} debug outlines from crosslinks
- fix ambiguous crosslinks: Build→"Coding ↓", Plan→"Enrich issues ↓"
- add missing "Edit build instructions ↑" backlink on Auto-address reviews
- normalize card title weight to text-sm font-semibold across all cards
- rename "Default" subcard to "Setup" with broader description
- fix Mentions description to imperative tone
- broaden Flags section description to cover built-in and custom
- remove useless fragments in ModesSection and ToolsSettings
- restructure Agent section: remove ConsoleSection wrappers, add sidebar indent support
Made-with: Cursor
* extract PR quick links as standalone card, consistent with issues
- PR quick links is now its own card under Reviews (was a sub-toggle inside Review PRs disabled state)
- Review PRs OFF sets prCreated="none" instead of auto-falling back to "links"
- Review PRs card hides sub-toggles when disabled (re-review/approve don't apply)
- Both PRs and Issues now have identical Quick links card structure
Made-with: Cursor
* update reviews screenshot with standalone quick links card
Made-with: Cursor
* polish dashboard UI: revert quick links to inline toggles, fix fonts and spacing
- revert standalone PR/issue Quick Links cards back to inline toggles inside
Review PRs and Enrich Issues cards (fixes prCreated state coupling bug)
- restore original font-medium card titles across all trigger/settings cards
- fix sidebar: add CONSOLE heading, remove nested indentation, remove truncation
- right-justify Enrich Issues mode dropdown, group description with label
- move instructions links inline with behavior descriptions
- replace text save indicators with icon spinner/checkmark
- standardize section title spacing, move footer below danger zone
Made-with: Cursor
* fix formatting for biome lint
Made-with: Cursor
* address PR review feedback: cleanup guard, shared util, wiki update
- clear ctx.toolState.review after read to prevent double-execution of postReviewCleanup
- forward authorPermission in safety-net re-review dispatch
- extract parseModeInstructions to utils/schemas/modeInstructions.ts
- update wiki/modes.md: remove stale v1/v2 language, fix dashboard layout
- add typecheck to pre-push hook
Made-with: Cursor
* add action typecheck to pre-push, fix exactOptionalPropertyTypes errors
Made-with: Cursor
* fix duplicate actuallyReviewedSha from rebase
Made-with: Cursor
* remove PR screenshots
Made-with: Cursor
* add label/textarea association for mode instruction accessibility
Made-with: Cursor
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Standardize orchestrator/subagent instructions on the MCP shell tool and format mode guidance as explicit checklists to make delegation flows easier to follow. Bump the action package version to 0.0.171 for this release.
Co-authored-by: Cursor <cursoragent@cursor.com>
* add workflows permission to git token and waitlist improvements
- add `workflows` to `InstallationTokenPermissions` type in both action and API token routes
- include `workflows: write` in the git token so agents can push workflow file changes
- add `githubFollowers` field to WaitlistSignup schema with migration
- add script to populate waitlist followers from GitHub API
- add frog-green-square-border logo asset
Co-authored-by: Cursor <cursoragent@cursor.com>
* improve CI, agent logging, token permissions, and delegation guardrails
- add format check and build step to root CI job
- standardize agent model/effort log lines across all agents
- fix GitHub App permissions types to match OpenAPI schema (workflows is write-only)
- improve delegation error message to prevent subagent recursion
- demote noisy OpenCode stderr to debug level
- add subagent delegation rules to resolved instructions
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix graphql partial error handling, update delegation message, add workflow_run fixtures
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove module-level env var throws that break CI build
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix logging bug and type hole from PR review
- use batch-local notFound counter so per-batch log doesn't undercount
- add workflows to WorkflowTokenPermissions so wire type matches what action sends
Co-authored-by: Cursor <cursoragent@cursor.com>
* lazy-init appOctokit to fix next build without env vars
Co-authored-by: Cursor <cursoragent@cursor.com>
* drop pnpm build from CI test workflow
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix delegate-effort test regex to match actual log format, disable fail-fast for agnostic tests
the test was matching `running \w+ with effort=auto` but the actual log
line from shared.ts is `» effort: auto`. also temporarily set
fail-fast: false on action-agnostic so all failures surface at once.
Co-authored-by: Cursor <cursoragent@cursor.com>
* disable fail-fast in action workflow too, relax ci.test.ts to match
both workflow files now use fail-fast: false for agnostic tests so all
matrix jobs run to completion. the ci consistency test now checks that
the two workflows agree rather than requiring true.
Co-authored-by: Cursor <cursoragent@cursor.com>
* restore fail-fast: true now that all agnostic tests pass
Co-authored-by: Cursor <cursoragent@cursor.com>
* skip agent tests in CI when agent harness file didn't change
adds action/test/changed-agents.sh which reads the PR diff (via
dorny/paths-filter) and outputs only agents whose harness file was
modified. the action-agents matrix now uses this dynamic list instead
of a hardcoded array, so e.g. a PR touching only cursor.ts runs 6
jobs instead of 30.
Co-authored-by: Cursor <cursoragent@cursor.com>
* update ci.test.ts to validate dynamic agent matrix
the test now checks that the matrix references the changes job output
and that changed-agents.sh correctly discovers all agents.
Co-authored-by: Cursor <cursoragent@cursor.com>
* parallelize action jobs and use claude canary fallback for shared changes
runs action-agents in parallel with action-agnostic after root/changes, and updates changed-agents logic so shared or non-harness action runtime changes run only claude while harness-specific edits run only those changed agents.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* console UI improvements and cleanup
- add verify workflow button and API endpoint for manual installation check
- move env var check into PromptBox as blocking overlay (hoisted to RepoConsole)
- extract FlagsCheatSheet modal, replace verbose flag hints everywhere
- add info popovers for repo setup / post-checkout script descriptions
- remove unused prAutoFixCiFailures schema fields and migration
- default mentionAllowNonCollaborator to disabled for safety on public repos
- update docs for triggers and getting started
Co-authored-by: Cursor <cursoragent@cursor.com>
* add diagnostic logging for push_branch bug investigation
temporary [push-debug] logs to trace why getPushDestination falls back
to origin/<localBranch> instead of using the correct remote branch name
for same-repo PRs.
Co-authored-by: Cursor <cursoragent@cursor.com>
* add git config diagnostic to verify original bug cause
Co-authored-by: Cursor <cursoragent@cursor.com>
* temporarily disable StoredPushDest to test git config path
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove diagnostic logging for push_branch investigation
verified that StoredPushDest fix works correctly on preview repo.
both the stored dest path and the git config fallback resolve to the
correct remote branch in the GitHub Actions environment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix formatting in AgentSettings and TriggersSettings
Co-authored-by: Cursor <cursoragent@cursor.com>
* pass derived env var state to PromptBox instead of raw secrets data
eliminates duplicated derivation logic between RepoConsole and PromptBox
by passing envVarMissing, envVarChecking, and agentKeyNames as props.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: prevent duplicate comment after PR review deletes progress comment
progressCommentId now uses three states: undefined (no comment yet),
number (active), null (deliberately deleted). After create_pull_request_review
deletes the progress comment, subsequent report_progress calls skip instead
of creating a new comment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* effort descriptions, test ordering, husky, docs images, typo fix
- rewrote delegation effort level descriptions to per-level breakdown
- action-agents now waits for action-agnostic; action-agnostic waits for root
- added husky + lint-staged (biome check --write on staged files)
- updated triggers docs images and triggers.mdx content
- fixed "figured" → "figures" typo on landing page
- updated pnpm-lock.yaml
Co-authored-by: Cursor <cursoragent@cursor.com>
* Commit
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
- Add `comment_type: "issue"` to `IssueCommentCreatedEvent` interface and
dispatch sites so agents can distinguish issue comments from PR review
comments
- Add dedicated "Progress reporting" section to system prompt making
`report_progress` the mandatory tool for sharing results
- Update `reply_to_review_comment` description to clarify it only works
for inline review comments on PR diffs, not issue comments
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* Scope installation token permissions in restricted mode
In restricted/disabled bash mode, the installation token is now scoped
to match the workflow's permissions block. This preserves fork push
capability while limiting what the agent can do with the token.
- Read workflow permissions from pullfrog.yml at runtime
- Pass permissions to API when acquiring installation token
- Clear OIDC env vars in restricted mode to prevent token minting
- Simplify setupGit by moving token resolution to main.ts
* Address review feedback: fail closed with default permissions
- Add restrictive default permissions (contents:read, pull_requests:read,
issues:read) as fallback when workflow permissions can't be read
- Add support for job-level permissions via GITHUB_JOB env var
- Fix misleading comment about token resolution in restricted mode
- Add documentation about fork PR checkout behavior
* Simplify to separate git/MCP tokens without workflow permission scoping
- gitToken: minimal contents:write only (assumed exfiltratable)
- mcpToken: full installation token (not exfiltratable via MCP tools)
- Remove workflowPermissions.ts - security-conscious users can pass
their own token via GH_TOKEN or inputs.token
- Add type-safe InstallationTokenPermissions to github.ts and API route
* Rename `write` permission to `push` and remove vestigial tool blocking
The `write` permission was previously used to block local file write tools
in agents. This was security theater since bash can write files anyway.
Now `push` only controls the git token scope:
- push: enabled → contents:write (can push commits)
- push: disabled → contents:read (read-only, can't push)
Changes:
- Rename `write` to `push` in action.yml, Prisma schema, and all TS types
- Remove vestigial write tool blocking from all agents (claude, cursor,
gemini, opencode, codex)
- Add data-preserving Prisma migration using RENAME COLUMN
- Update UI: "Write files" → "Git push" with updated description
* add PID namespace isolation for bash sandbox
when running in CI, attempts to use unshare --pid to create a new PID
namespace for bash subprocesses. this prevents the /proc/$PPID/environ
attack where a malicious command could read secrets from the parent
process's environment.
the protection works by:
1. creating a new PID namespace (subprocess becomes PID 1)
2. mounting fresh /proc showing only sandbox PIDs
3. parent PIDs become invisible (PPID = 0, /proc/0 doesn't exist)
combined with filterEnv(), this provides complete protection against
/proc-based secret theft. falls back gracefully if namespaces aren't
available.
includes test script to verify the protection works.
* add PID namespace test to CI workflow
tests whether unshare --pid works on GHA runners out of the box,
and if not, whether enabling via sysctl helps. also runs the
pidNamespace.ts test to verify the full protection.
* fix pnpm setup and add procIsolation agent test
- fix pnpm/action-setup by specifying package_json_file path
- add procIsolation crossagent test that has agent attempt to
read secrets via /proc/$PPID/environ
- add procIsolation to CI test matrix
* add pid-namespace test job to main workflow
this job tests unshare --pid capabilities on GHA runners and runs
the pidNamespace.ts adhoc test to verify /proc isolation works
* test bubblewrap's sysctl approach for enabling namespaces
- write to /etc/sysctl.d/99-userns.conf and run sysctl --system
- try aa-complain on unshare binary
- more detailed diagnostics
* fix pidNamespace test and add sudo-unshare fallback for GHA
- fix reference error in pidNamespace.ts (renamed function but didn't update calls)
- add sudo-unshare as fallback method for GHA runners where unprivileged
namespaces are blocked but sudo is available
- update bash.ts to detect and use sudo unshare when unprivileged fails
* consolidate security docs and document PID namespace isolation
- update security.md with current implementation details
- document sudo unshare fallback for GHA runners
- add testing instructions for local Docker and CI
- add "Further Exploration" section with Landlock and path validation ideas
- delete bash-sandbox.md and landlock.md (consolidated into security.md)
* move procIsolation test to adhoc folder
the procIsolation test requires PID namespace capabilities that aren't
available in the Docker test environment. moved to adhoc/ so it's excluded
from default test runs and can be run explicitly when needed (e.g. via
the pid-namespace CI job or locally with --privileged docker).
* fix Docker test environment for PID namespace isolation
- add CI and GITHUB_ACTIONS to testEnvAllowList so sandbox detection runs
- add --privileged to Docker run for PID namespace support (unshare)
this fixes the test environment to properly test the sandbox. in production,
the action runs directly on GHA runner where sudo unshare works.
* fix getJobToken() to work in test environment
add fallback to GH_TOKEN and GITHUB_TOKEN when INPUT_TOKEN is not set.
this allows tests to run without requiring workflow-level token input.
the token resolution order is:
1. INPUT_TOKEN (from workflow `with: token:`)
2. GH_TOKEN (external token override)
3. GITHUB_TOKEN (pre-acquired in tests or from GHA env)
* security: filter secrets from all subprocess environments
- extract filterEnv() to shared utils/secrets.ts
- make $() utility filter secrets by default (git, npm, etc. don't need them)
- disable git hooks via core.hooksPath to prevent hook-based exfiltration
- git auth uses token embedded in URL, not env vars
this prevents malicious git hooks, npm postinstall scripts, and other
code execution vectors from exfiltrating GITHUB_TOKEN and API keys.
* docs: clarify defense-in-depth security model
update security.md to explain why BOTH layers are required:
- filterEnv(): cleans child's own /proc/self/environ
- PID namespace: hides parent's /proc entries
PID namespace alone isn't sufficient - with --mount-proc, the child
becomes PID 1, so /proc/1/environ is the child's OWN environment.
without filterEnv(), secrets would still be accessible there.
* add procSandbox crossagent test for PID namespace security
- add crossagent/procSandbox.ts: security test that instructs agent to try
various /proc attack vectors and validates sandbox blocks them
- update wiki/security.md: document PID namespace isolation details, add
verification commands, explain why sudo inside sandbox doesn't break security
- update docker.ts: use node:24 with sudo for GHA-like test environment
- update instructions.ts: allow disabling security messaging for pentests
- clean up adhoc test files (procIsolation.ts, securityAudit.ts)
the procSandbox test sets SANDBOX_TEST_TOKEN (matches *_TOKEN filter), gives
the agent explicit hints about attack vectors (/proc, sudo, nsenter, etc.),
disables prompt-based mitigations, and validates the secret is never leaked.
* move procSandbox test to agnostic/ (runs with one agent)
* WIP
* docs: add agent testing guide (pnpm play, Docker, pentesting)
* docs: add CI details to agent testing guide
* docs: add interesting findings and gotchas from pentesting
* improve test fidelity: auto-set CI=true, verify sandbox active
- docker.ts: always set CI=true in container so sandbox activates
- docker.ts: skip sudo/user setup if already done (faster reruns)
- bash.ts: export getSandboxMethod() for diagnostics
- bash.ts: add debug log when sandbox disabled
- procSandbox.ts: add sandbox_was_active check to catch vacuous passes
the CI=true change is critical: without it, PID namespace isolation
is skipped and security tests pass without actually testing anything.
* docs: update agent-testing.md with CI=true auto-set note
* docs: clarify log format is agent-specific
* fix git auth, simplify MCP tools, add adversarial tests
- fix gitWithAuth to use Basic auth format (Bearer doesn't work with git's http.extraheader)
- fix token.ts: push:restricted now correctly gets contents:write
- fix github.ts: pass permissions when acquiring installation tokens locally
- remove commit_files and create_branch MCP tools (redundant, don't require credentials)
- remove containsSecrets() - trivially bypassable, not sound security
- add agnostic adversarial tests: pushDisabled, pushRestricted, tokenExfil
- update instructions.ts to clearly list available git MCP tools
- add wiki/git.md documenting credential isolation and push permission tiers
- update wiki/docker.md with custom image considerations
- update wiki/agent-testing.md with adversarial testing patterns
* fix type errors after rebase
- change ResolveTokensParams.push from ToolPermission to PushPermission
- use tags: ["agnostic"] instead of agnostic: true in test files
* fix cleanup permission error in sandbox tests
when sandbox isolation is enabled (CI=true), files created by the unshare
subprocess may have different ownership, causing rmSync to fail with EACCES.
this error in the finally block was overriding the test's success result.
fix: wrap cleanup in try-catch and fall back to sudo rm if rmSync fails.
* Add adhoc
* Handle git config/remote bypasses
* add git hooks protection and simplify ToolState
- disable git hooks in restricted mode via -c core.hooksPath=/dev/null
- add gitHooks adversarial test to verify hook protection works
- unify prNumber/issueNumber into single issueNumber field
- add pushUrl to ToolState for push validation
- add generateTestMarker() for simpler single-agent test markers
- export SENSITIVE_PATTERNS and isSensitiveEnvName from secrets.ts
- remove redundant pidNamespace.ts (duplicated by procSandbox.ts)
- update documentation
* harden $git() auth: subcommand whitelist, binary tamper detection
- rename gitWithAuth() to $git() with explicit subcommand first arg
- restrict to "fetch" | "push" at type level (filters don't run for these)
- resolve git binary path at startup via resolveGit(), sha256 fingerprint
- verify hash before each $git() call to detect binary replacement
- rename disableHooks to restricted for cleaner semantics
- document filter exfiltration attack and empirical verification in wiki
* remove redundant pid-namespace CI job
the PID namespace isolation testing is now handled by
action/test/agnostic/procSandbox.ts via pnpm runtest agnostic
* fix push_branch for new branches and improve token leak detection
- getPushDestination now falls back to origin/<branch> when @{push}
is not configured (happens for new branches created locally)
- gitPerms validator now checks for actual token patterns instead
of matching "x-access-token" string in test instructions
* use kebab-case for test names
* simplify shell env API: "restricted" | "inherit" | object
replace passFullEnv boolean with cleaner env option that accepts:
- "restricted" (default): filterEnv() to prevent secret leakage
- "inherit": full process.env
- object: custom env merged with restricted base
* share EnvMode and resolveEnv between shell.ts and bash.ts
move shared env resolution logic to secrets.ts
* add env option to bash tool (default: restricted)
* delete agent-testing.md (renamed to adversarial.md)
* Add checkout tests
* reframe githooks test prompt to avoid claude safety refusal
claude was refusing to execute the test because the prompt used words
like "malicious" and "security testing". reframed as a debugging task
with innocuous env var name (TESTING_DEBUG_TAG) per adversarial.md guidance.
Co-authored-by: Cursor <cursoragent@cursor.com>
* clean up verbose token acquisition logs
move logging responsibility to call sites which have better context
(git token vs MCP token). remove redundant intermediate OIDC logs
and unused "(permission-scoped)" suffix.
Co-authored-by: Cursor <cursoragent@cursor.com>
* isolate agnostic tests with matrix strategy, fix .pullfrog-env secret leak
- split action-agnostic into per-test matrix jobs for isolated logs and filesystems
- only write explicitly opted-in env vars to .pullfrog-env via fileAgentEnv
(fixes token-exfil test where claude found SANDBOX_TEST_TOKEN on disk)
- mcpmerge test opts in via fileAgentEnv for cursor's repo-level MCP fallback
Co-authored-by: Cursor <cursoragent@cursor.com>
* remove env parameter from bash tool to prevent agents bypassing filterEnv
the bash tool exposed an `env` parameter accepting "restricted" | "inherit"
which allowed agents to pass env: "inherit" and see all secrets including
SANDBOX_TEST_TOKEN, bypassing the restricted environment filtering entirely.
env mode is now determined internally (always restricted).
Co-authored-by: Cursor <cursoragent@cursor.com>
* use pullfrog/test-repo for push tests to stop polluting main repo
push tests were creating branches and tags on pullfrog/app directly.
now all push tests (push-restricted, push-disabled, push-enabled,
git-permissions) target pullfrog/test-repo instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
* use pullfrog/test-repo for all tests, not just push tests
no test should clone or operate on pullfrog/app directly.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix token scoping for test-repo and bash timeout defaults
- acquireTokenViaOIDC now includes GITHUB_REPOSITORY repo in token
scope so push tests work against pullfrog/test-repo
- bash tool default timeout: 120s -> 30s, cap: 600s -> 120s
- activity timeout: 30s -> 60s
- prevents hung bash commands (e.g. find /) from killing the agent
via activity timeout
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add utils for r2 upload
* Add the tool and new routes
* fix auth issue
* sign headers
* add comment
* use our own API key to auth signed uploads
* Restructure things slightly
* tweak
* tweak
* add comments
* tweak
* revert a thing
* twaek
* drop mime type filtering
* new incarnation of mime type filtering
* jsut allow all octet-streams
* simplify further
* tweak
* update lockfile
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
- Move prompt construction logic from action-side to server-side (webhook handler and trigger page)
- Include issue/comment body in USER PROMPT only if @pullfrog was tagged (checked server-side using containsTriggerPhrase)
- Add repoInstructions as separate REPO-LEVEL INSTRUCTIONS section in FULL prompt
- Macro-expand repoInstructions server-side before sending to action
- Trigger page never includes body (manual triggers)
- Remove redundant customInstructions field (now combined into prompt server-side)
files changed:
- action/external.ts: add repoInstructions to WriteablePayload, remove customInstructions
- action/utils/payload.ts: add repoInstructions to JsonPayload schema, remove customInstructions
- action/utils/repoSettings.ts: add repoInstructions to RepoSettings interface
- action/utils/instructions.ts: use payload.prompt directly, add repo section to full prompt, add repo field to ResolvedInstructions
- utils/webhooks/handleWebhook.ts: check @pullfrog tag and include body if tagged, macro-expand repoInstructions
- app/trigger/[owner]/[repo]/[number]/page.tsx: macro-expand repoInstructions (never include body)
Colin McDonnell