3 Commits

Author SHA1 Message Date
David Blass 6b18b6730b live todo tracking, collapsible task list in final progress, hide set_output outside standalone (#492)
* fix false "without reporting progress" error + live todo tracking

clean up orphaned progress comments when review is skipped or only
set_output is used, preventing the false positive in handleAgentResult.

parse todowrite events from OpenCode's NDJSON stream and render a
live markdown checklist in the PR progress comment (2s debounce).
agent's explicit report_progress always takes priority.

Made-with: Cursor

* fix contradictory review/progress prompting

align Review and IncrementalReview mode prompts with their guidance —
mode prompts said "always submit" while guidance said "skip if clean."
remove the empty-approval submission that was silently dropped by the
tool. make progress comment lifecycle explicit: created on first call,
updated in place, removed after review submission.

Made-with: Cursor

* centralize todo tracking into shared TodoTracker module

extract inline todo tracking logic (~95 lines) from opentoad.ts into
action/utils/todoTracking.ts. the tracker is created once in main.ts
and passed to agents via AgentRunContext.todoTracker, making it
agent-agnostic and reusable for future agent implementations.

Made-with: Cursor

* fix todoTracker optional type to match file convention

add | undefined to todoTracker in AgentRunContext, matching every
other optional property in the same interface.

Made-with: Cursor

* instruct agents to always maintain a task list for live progress

system prompt now tells agents to create an internal task list at
the start of every run. the tracker renders it to the progress
comment automatically. report_progress is reserved for final
results only — no more intermediate "Checking..." messages that
cancel the tracker and leave stale text on the comment.

Made-with: Cursor

* require report_progress summary at end of every run

agents must always call report_progress with a final summary —
the completed task list should never be the end state of the
progress comment. updated all review mode prompts to call
report_progress after submitting (or not submitting) a review.

Made-with: Cursor

* keep progress comment after review with final summary

stop deleting the progress comment after review submission —
the agent now always calls report_progress with a summary at
the end, and that summary should persist as a record of what
was done.

Made-with: Cursor

* harden stranded progress comment cleanup

- main.ts: detect when tracker was last writer (agent never called
  report_progress) and delete the stranded checklist instead of
  leaving it as the final comment state
- postCleanup.ts: expand stuck-comment detection to also catch
  stranded todo checklists (regex match for checklist patterns)
  when the process is killed before normal cleanup runs
- modes.ts + selectMode.ts: add report_progress step to Summarize
  and SummaryUpdate modes (only modes that were missing it)

Made-with: Cursor

* fix stale comments, typo, and build mode redundancy

- comment.ts: update deleteProgressComment docstring and inline comment
  to reflect current usage (stranded-comment cleanup, not post-review)
- modes.ts: merge duplicate report_progress steps (8 + 10) into single
  step 9, fix "optimizatfixons" typo
- wiki/post-cleanup.md: document checklist detection regex

Made-with: Cursor

* collapsible completed todos in final progress, hide set_output outside standalone mode

- add renderCollapsible() to TodoTracker, append completed task list as
  <details> section when agent calls report_progress
- cancel tracker after agent's final report_progress so it doesn't
  overwrite with raw checklist
- conditionally register SetOutputTool only in standalone mode or when
  output_schema is provided
- remove unconditional set_output instruction from orchestrator task section
- update Summarize/SummaryUpdate mode guidance to not reference set_output

Made-with: Cursor

* show completion count in collapsible task list summary

Made-with: Cursor

* only count completed (not cancelled) in collapsible task list summary

Made-with: Cursor

* reinforce concise summary prompting across system prompt, modes, and tool description

Made-with: Cursor

* address review feedback: wasUpdated bypass, tracker false-positive, race condition

- remove wasUpdated=true from cleanup paths so handleAgentResult correctly
  detects genuinely silent runs
- add hadProgressComment to ToolState as immutable snapshot for the safety check
- use todoTracker.hasPublished instead of enabled for stranded-comment cleanup
- serialize onUpdate calls via inflightPromise chain with post-cancel guard
- add settled() to wait for in-flight updates before writing final summary

Made-with: Cursor

* address round-2 review: hasPublished after success, finalSummaryWritten flag

- set hasPublished only after onUpdate resolves (not before) so failed
  writes are not counted as published
- add finalSummaryWritten flag to ToolState, set after successful
  non-plan reportProgress; decouple cleanup detection from
  todoTracker.enabled so it survives API failures where cancel() ran
  but the write didn't succeed

Made-with: Cursor
2026-03-25 19:35:31 +00:00
Colin McDonnell 6d25adfd1a Agent & model refactor (#478)
* agent & model refactor with ASKPASS git auth, UI restructure, clerk v7

Made-with: Cursor

* fix stale agent/effort refs, add tests for askpass + model resolution

- reviewCleanup.ts: payload.agent -> payload.model, remove effort
- selectMode.ts PlanEdit: remove delegation/subagent/effort references
- pullfrog.yml.ts: update env vars (drop GOOGLE_API_KEY/CURSOR_API_KEY,
  add GOOGLE_GENERATIVE_AI_API_KEY/XAI_API_KEY/MOONSHOT_API_KEY/OPENCODE_API_KEY)
- FlagsSettings/RepoInstructionsSection: remove stale effort/timeout copy
- new: gitAuthServer.test.ts (10 tests — lifecycle, token delivery, tamper detection, script gen)
- new: agent.test.ts (4 tests — default opentoad, AGENT_OVERRIDE, invalid override)
- new: models.test.ts (19 tests — parseModel, resolution, registry invariants)
- update models.dev snapshot

Made-with: Cursor

* fix changed-agents.sh to filter legacy agent files from CI matrix

legacy agent files (claude.ts, codex.ts, etc.) are @ts-nocheck and not
exported from index.ts. changed-agents.sh now reads index.ts imports to
build the active agent set and treats changes to inactive files as
non-agent changes (opentoad canary only).

Made-with: Cursor

* remove MCP file tools, old agent harnesses, and obsolete security tests

ASKPASS-based git auth makes the old MCP file tool security layer unnecessary:
- token never in subprocess env, so symlink/gitattributes/hook attacks can't exfiltrate it
- agents now use native file tools (OpenCode builtin read/edit)

deleted:
- action/mcp/file.ts (file_read, file_write, file_edit, file_delete, list_directory)
- action/mcp/index.ts (dead re-export)
- agent harnesses: claude.ts, codex.ts, cursor.ts, gemini.ts, opencode.ts
- opencode-runner.ts (inlined into opentoad.ts)
- security tests that validated MCP file tool restrictions
- commented-out three-step review flow (~300 lines)
- sanitizeSchema/wrapSchema dead code from mcp/shared.ts
- OPENCODE_MODEL_MINI/MAX env vars (effort-level model overrides removed)

updated test prompts to use generic file ops instead of MCP tool names.
restored pkg-json-scripts + requirements-txt-attack (test --ignore-scripts defense).

Made-with: Cursor

* bump actions/checkout v4 → v6 (node 24)

node 20 actions deprecated june 2, 2026.

Made-with: Cursor

* temporarily disable fail-fast on agnostic tests to debug checkout@v6

Made-with: Cursor

* re-enable fail-fast on agnostic tests

Made-with: Cursor

* fix test token mismatch: mint OIDC tokens scoped to target repo

CI tests override GITHUB_REPOSITORY to pullfrog/test-repo but inherit
the runner's GITHUB_TOKEN (scoped to pullfrog/app), causing 401s on
every run-context fetch. Clear GITHUB_TOKEN in the test subprocess so
ensureGitHubToken() mints a properly scoped token via OIDC.

Also centralizes the default GITHUB_REPOSITORY in runAgentStreaming
instead of repeating it in every test file, and fixes preview-cleanup
to remove workers from all queues (not just name-matching ones).

Made-with: Cursor

* fix ensureGitHubToken to try OIDC when app credentials are absent

ensureGitHubToken only attempted token minting when GITHUB_APP_ID and
GITHUB_PRIVATE_KEY were set. In CI, OIDC is available but app creds
aren't exposed — so the guard prevented minting entirely.

Made-with: Cursor

* dead code cleanup: remove remnants of deleted agents, file tools, effort system

remove unused @anthropic-ai/claude-agent-sdk and @openai/codex-sdk deps,
orphaned file-tool security tests, dead GEMINI_MODEL passthrough, stale
opencode-runner wiki refs, deleted test file references, and MCP file tool
docs. rename docs/effort → docs/models. fix vitest setup: move dotenv to
globalSetup (runs once before forks instead of per-file, 19s → 200ms).

Made-with: Cursor

* address review feedback: remove dead code, update stale references

- remove AGENT_OVERRIDE (only opentoad exists)
- remove shellToolName plumbing (always restricted shell)
- bump action version to 0.0.179
- remove CURSOR_API_KEY from all workflows/configs
- remove OPENCODE_MODEL_MINI/MAX from workflows/docs
- delete wiki/effort.md, rewrite docs/effort.mdx as "Models"
- rewrite wiki/modes.md: orchestrator/subagent → single agent
- simplify flag system: drop builtin flag extraction (debug, effort,
  timeout, agent), keep custom flag replacement only
- reserve all legacy flag names to prevent custom flag conflicts

Made-with: Cursor

* regenerate lockfile after removing claude-agent-sdk and codex-sdk

Made-with: Cursor

* fix import ordering, add lockfile check to pre-push hook

Made-with: Cursor

* remove dead debug payload field, stale packageExtensions

Made-with: Cursor

* merge proc-sandbox and token-exfil into a single test

proc-sandbox and token-exfil were duplicative — both tested that
SANDBOX_TEST_TOKEN couldn't be exfiltrated. consolidated into
token-exfil with shell:restricted (which actually exercises filterEnv)
and the /proc attack vector hints from proc-sandbox.

Made-with: Cursor

* fix wiki adversarial.md to match actual tokenExfil validator

Made-with: Cursor
2026-03-12 05:22:51 +00:00
David Blass 5bcfae990a restructure dashboard UI, add mode instructions, post-review follow-up dispatch (#453)
* add mode instructions and restructure dashboard sidebar

- add modeInstructions JSONB field to Repo model for per-mode user instructions
- thread modeInstructions through settings API, run-context API, RepoSettings, ToolContext, and selectMode runtime
- merge user-defined mode instructions with hardcoded orchestrator guidance, with IncrementalReview inheriting from Review
- reduce visible built-in modes from 7 to 4 (Build, Review, Plan, Fix) with editable Instructions textareas
- add TRIGGERS group header to sidebar above Mentions, Pull requests, Issues
- add wiki/modes.md documenting triggers and modes conceptual model

Made-with: Cursor

* fix leaping comment deletion and address review feedback

- wrap post-createReview operations in try/finally so deleteProgressComment
  runs even when updateReview or reportReviewNodeId throws
- add parseModeInstructions runtime guard to filter non-string values
  from the JSONB field before passing to buildOrchestratorGuidance
- add useEffect sync for localInstructions when props change
- guard onBlur to skip save when instructions haven't changed
- update wiki/modes.md to reflect V2 is implemented (no longer "proposed")

Made-with: Cursor

* harden review cleanup, fix type cast, stabilize mode instructions state

- wrap deleteProgressComment in try/catch inside finally to prevent masking original errors
- replace `as Record<string,string>` cast with runtime parseModeInstructions + useMemo
- fix wiki dual-prompt table to reflect mode.prompt fallback status

Made-with: Cursor

* fix wiki tense and heading ambiguity from PR review

Made-with: Cursor

* fix review "edited" badge by using pending review + submit flow

create review as PENDING first (no event/body), build the footer with
the now-known review ID, then submitReview with the full body. single
atomic publish — no updateReview edit needed.

Made-with: Cursor

* add post-agent follow-up re-review dispatch

After the agent exits, check if PR HEAD moved past the reviewed commit
and dispatch a follow-up re-review. This closes the gap where push
webhooks are suppressed during in-flight reviews.

Made-with: Cursor

* add silent flag to follow-up re-review dispatch

Made-with: Cursor

* restructure dashboard for consistency and clarity

- consolidate tools into single grouped card (was 4 separate cards)
- merge coding + autofix CI into one section
- remove redundant trigger section descriptions
- add bidirectional crosslinks between modes and triggers
- inline instruction links (review/plan/build) into descriptions
- add save status indicators to all sections
- restructure flags with grouped built-in/custom cards
- flatten sidebar (remove dividers and group headers)
- tighten all descriptions

Made-with: Cursor

* update PR screenshots for new dashboard layout

Made-with: Cursor

* extend review context inline instead of dispatching new workflow

when commits are pushed during a review, the agent now handles them
inline: create_pull_request_review detects HEAD movement, returns
instructions to pull and review the incremental diff, and the agent
submits a second review covering only the new changes. this avoids
the cost of spinning up a full new workflow run.

also fixes a bug where reviewedSha was set to the submission HEAD
(current) rather than the checkout HEAD (what was actually reviewed),
which caused commits pushed between checkout and submission to be
silently missed by postReviewCleanup.

the workflow dispatch is kept as a safety net for agent timeout/error.

Made-with: Cursor

* polish dashboard UI: fix debug markers, crosslinks, title consistency, descriptions

- remove all red debug borders/labels and CM component
- remove all inline style={{}} debug outlines from crosslinks
- fix ambiguous crosslinks: Build→"Coding ↓", Plan→"Enrich issues ↓"
- add missing "Edit build instructions ↑" backlink on Auto-address reviews
- normalize card title weight to text-sm font-semibold across all cards
- rename "Default" subcard to "Setup" with broader description
- fix Mentions description to imperative tone
- broaden Flags section description to cover built-in and custom
- remove useless fragments in ModesSection and ToolsSettings
- restructure Agent section: remove ConsoleSection wrappers, add sidebar indent support

Made-with: Cursor

* extract PR quick links as standalone card, consistent with issues

- PR quick links is now its own card under Reviews (was a sub-toggle inside Review PRs disabled state)
- Review PRs OFF sets prCreated="none" instead of auto-falling back to "links"
- Review PRs card hides sub-toggles when disabled (re-review/approve don't apply)
- Both PRs and Issues now have identical Quick links card structure

Made-with: Cursor

* update reviews screenshot with standalone quick links card

Made-with: Cursor

* polish dashboard UI: revert quick links to inline toggles, fix fonts and spacing

- revert standalone PR/issue Quick Links cards back to inline toggles inside
  Review PRs and Enrich Issues cards (fixes prCreated state coupling bug)
- restore original font-medium card titles across all trigger/settings cards
- fix sidebar: add CONSOLE heading, remove nested indentation, remove truncation
- right-justify Enrich Issues mode dropdown, group description with label
- move instructions links inline with behavior descriptions
- replace text save indicators with icon spinner/checkmark
- standardize section title spacing, move footer below danger zone

Made-with: Cursor

* fix formatting for biome lint

Made-with: Cursor

* address PR review feedback: cleanup guard, shared util, wiki update

- clear ctx.toolState.review after read to prevent double-execution of postReviewCleanup
- forward authorPermission in safety-net re-review dispatch
- extract parseModeInstructions to utils/schemas/modeInstructions.ts
- update wiki/modes.md: remove stale v1/v2 language, fix dashboard layout
- add typecheck to pre-push hook

Made-with: Cursor

* add action typecheck to pre-push, fix exactOptionalPropertyTypes errors

Made-with: Cursor

* fix duplicate actuallyReviewedSha from rebase

Made-with: Cursor

* remove PR screenshots

Made-with: Cursor

* add label/textarea association for mode instruction accessibility

Made-with: Cursor

---------

Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
2026-03-11 04:24:09 +00:00