Update Build, AddressReviews, and Prompt modes to ensure tests are run
BEFORE committing and pushing code. This prevents redundant workflow
triggers when tests fail and need fixes.
Previously, the Build and Prompt modes would:
1. Make code changes
2. Commit and push
3. Test (oops, too late!)
Now all modes follow the correct order:
1. Make code changes
2. Test (if tests fail, fix and repeat)
3. Commit and push
This addresses the inefficiency observed in #268 where the agent pushed
code before verifying it worked, then had to fix and push again.
Closes#273
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* Replace Date.now() with performance.now() for duration measurements
- Import performance from node:perf_hooks in all affected files
- Update Timer and ThinkingTimer classes to use performance.now()
- Update activity tracking (markActivity, getIdleMs) to use performance.now()
- Update cache duration measurements to use performance.now()
- Update agent execution timing (cursor, opencode) to use performance.now()
- Update subprocess execution timing to use performance.now()
- Update API performance monitoring to use performance.now()
- Update prep phase timing to use performance.now()
- Update timer.test.ts to mock performance.now() instead of Date.now()
Benefits:
- Monotonic clock immune to system clock adjustments
- Higher precision (microsecond vs millisecond resolution)
- Purpose-built for performance measurement
Fixes#245
* fix lint.
* Round float durations to integers in logging
Preserve original behavior by rounding performance.now() float values
to integers when displaying/logging millisecond durations.
* fix lint.
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Robin Tail <robin_tail@me.com>
* feat(agents): add thinking time logging between tool calls
Adds a ThinkingTimer utility that tracks the gap between tool results and
the next tool call. When the gap exceeds 3 seconds, it logs the duration
with a stopwatch emoji (⏱️ 4.2s).
Uses performance.now() for high-resolution timing and Intl.NumberFormat
for rendering duration in seconds with optional fraction digits.
Integrated across all 5 agents: Claude, Codex, Cursor, Gemini, OpenCode.
Closes#127
* fix: adjusting tests for mocking performance.now.
* fix: reducing diff for claude.
* fix: rm unused args for claude.
* rm unused args for codex.
* fix: rm unused args for gemini.
* fix: rm unused args for opencode.
* mv THINKING_THRESHOLD.
* rev: I decided to pospone node:perf_hooks integration since it requires more comprehensive refactoring.
* fix: using Intl unit formatting.
* tests for ThinkingTimer.
* fix: narrow unit.
* fix: making durationFormatter a class instance property since using one agent per run.
* fix: inverting condition in markToolCall.
* thinking timer improvements and fix actions/checkout v6 auth
- thinking timer: use » chevron and "thought for X seconds" format
- thinking timer: add debug timestamps for sanity checking
- demote PID namespace isolation logs to debug
- remove redundant "setting up git authentication" log
- fix duplicate Authorization header with actions/checkout v6: clean up
includeIf credential entries that v6 persists via external config files
Co-authored-by: Cursor <cursoragent@cursor.com>
* standardize tool call log prefix to » double chevron
Co-authored-by: Cursor <cursoragent@cursor.com>
* update timer tests for new thinking log format
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Robin Tail <robin_tail@me.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* flatten lifecycle hooks into RepoSettings string fields
replace the separate LifecycleHook model with setupScript and
postCheckoutScript string fields directly on RepoSettings. move the UI
into the Agent settings section alongside environment variables and
custom instructions. delete the standalone lifecycle-hooks API route,
component, and schema since the existing settings PATCH endpoint
handles the new fields automatically.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: pass env to lifecycle hook spawn so scripts can use package managers
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Update Build and Prompt mode instructions to explicitly reference
`issue_number` from EVENT DATA and instruct agents to include
"Closes #<issue_number>" in PR bodies when working in the context
of an issue (where `is_pr` is not true).
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* feat: post "Leaping..." comment immediately without polling GitHub API
This change makes the initial comment response much faster by avoiding
the expensive GitHub API polling that was waiting for workflows to
dequeue (up to 12s+ in some cases).
New architecture:
1. Create WorkflowRun record BEFORE dispatching (no runId yet)
2. Post "Leaping into action..." comment with shortlink URL immediately
3. Dispatch workflow and return
4. workflow_run.requested webhook fills in runId when GitHub dequeues
Shortlink redirect at /api/workflow-run/[id]/logs:
- If runId available: redirects to GitHub workflow run
- If runId null: shows polling page that checks DB every 1.5s
Changes:
- Make `runId` optional on WorkflowRun model (filled in via webhook)
- Add `workflow_run` to expected webhook events
- Add handler for workflow_run.requested to update DB with runId
- Create /api/workflow-run/[id]/logs shortlink redirect route
- Refactor triggerWorkflow.ts to use eager comment pattern
- Update trigger page to use new pattern
Closes#141
* chore: add migration for nullable runId in WorkflowRun
* fix: rm dead code.
* fix: Adjusting the issueNumber prop usage comment.
* fix: shortening and JSDoc for createWorkflowRunRecord.
* fix: Reducing diff, reducing confusion on naming the id.
* Revert "fix: Adjusting the issueNumber prop usage comment."
This reverts commit 34d87c2f8bc58782a53ce5eb14a40935232a4924.
* refactor: reuse buildShortlinkUrl in trigger page
* fix: Reducing confusion on param naming.
* fix: shorter JSDoc.
* Apply suggestion from @RobinTail
* chore: remove unnecessary JSDoc comment from buildShortlinkUrl
* Revert "chore: remove unnecessary JSDoc comment from buildShortlinkUrl"
This reverts commit 491ba6ba3f31c874c9f391871739efa50adb0446.
* fix: confusing naming of var.
* fix: redundant 'let'.
* refactor: move route from `/api/workflow-run/[id]/logs` to `/api/workflow-run-logs/[id]`
Avoids confusion with existing `/api/workflow-run/[runId]` route which uses
GitHub's runId, whereas this new route uses the internal WorkflowRun record id.
* chore: remove old route directory
* refactor: reuse `buildShortlinkUrl()` with `shouldPoll` param
* refactor: add script prop to generateLeapingLoaderHtml
Instead of string-replacing to inject scripts, the function now
accepts an optional script prop that gets wrapped in <script> tags.
* mv script into new LeapingLoaderHtmlProps.
* refactor: extract `buildGithubUrl` helper to avoid repetition
* fix: shorening.
* fix: More clear subtitle.
* refactor: reuse `WORKFLOW_FILENAME` from `app/globals.ts`
* fix: shortening.
* feat: add integrity_id for reliable workflow matching
Pass WorkflowRun record id as integrity_id when dispatching workflows.
The webhook handler parses integrity_id from display_title (via run-name)
for reliable matching, with fallback to repo/owner lookup when missing.
Note: workflow template changes (.github/workflows/pullfrog.yml) need to be
applied manually as the GitHub App lacks workflows permission.
* Revert "feat: add integrity_id for reliable workflow matching"
This reverts commit dbc601233a0dd85ac5f0d608a221e7015e265aaa.
* Add todo for consideration later.
* docs: add plan for action-initiated workflow run correlation
Addresses review feedback requesting research into secure alternatives
to exposing HOOKDECK_API_KEY. Proposes leveraging existing OIDC token
exchange to pass WorkflowRun record ID and correlate with run_id.
* Revert "docs: add plan for action-initiated workflow run correlation"
This reverts commit b9279d0e99db4d85e4144675634afa941858333a.
* FEAT: Add optional integrity_id input, used by run-name, set with partial record id, read by handler for lookup.
* Add integrity_id to app/trigger/[owner]/[repo]/[number]/page.tsx
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* feat: Extracting INTEGRITY_ID_LENGTH.
* Add integrity_id to the workflow files of the repo.
* fix: Use Vercel Preview deployment URL into account in buildShortlinkUrl().
* fix: add missing `.rest` prefix in Octokit API call
* revert: remove unnecessary escaping of backticks in comment
* fix: add polling timeout and wrap DB update in try-catch
- Add 3-minute timeout to polling page to prevent indefinite polling
- Wrap updateWorkflowRunComment in try-catch to prevent orphaned comments
* feat: restore job-level deep linking for workflow runs
Adds jobId to WorkflowRun model and captures the first job ID via
listJobsForWorkflowRun() when the workflow_run_in_progress event fires.
The shortlink redirect now appends /job/{jobId} when available, providing
a direct link to the job rather than just the workflow run.
* fix: handle only `workflow_run.in_progress` to avoid race condition
Combine the handling of `runId` and `jobId` into a single update when
`workflow_run.in_progress` fires, avoiding the race condition where
`in_progress` could arrive before `requested` was processed.
* Renae integrity_id -> name
* Clean up
* Clean up
* Shorter timeout
* Add fallback
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Robin Tail <robin_tail@me.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
- Build mode: rewrote steps 8-9 to consolidate PR creation logic into step 8
with explicit default/branch-only behaviors, removing the false claim that
create_pull_request is needed for commit attribution
- Prompt mode: updated step 2 with the same conditional PR creation logic
Fixes#84
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
* add api key existence check when selecting agent
- create getSecretNames utility to fetch GitHub Actions secret names
- add /api/repo/[owner]/[repo]/secrets endpoint to check secrets
- update AgentSettings to fetch and display secret validation status
- show green check when required API key exists
- show amber warning when required API key is missing
- show loading state while checking secrets
* Add API key checking
* Fix null agent test
* Tweaks
* Switch to getrepoorgsecretes
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* fix: prevent log.writeSummary from overwriting reportProgress content
The run summary was showing logs instead of the final reportProgress content
because log.writeSummary() was called after reportProgress. Now
log.writeSummary() checks if the summary was already overwritten by
reportProgress and skips if so.
Fixes#86
* refactor: replace dynamic import with static import in cli.ts
Replace unnecessary dynamic import of wasSummaryOverwritten with
static import. No circular dependency exists since comment.ts doesn't
import from cli.ts.
* Fix run summary writing
---------
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
* implement github suggestion format instructions
add instructions for agents to use github's suggestion format (```suggestion blocks) when providing code suggestions in comments. this enables one-click apply for suggested changes.
updated:
- action/mcp/review.ts: added suggestion format guidance to create_pull_request_review tool description and comment body parameter
- action/mcp/comment.ts: added suggestion format guidance to all comment tools with clarification that suggestions only work on pr line-level review comments
- action/modes.ts: added detailed example in review mode and reminder in address reviews mode
fixes#70
* Address PR review feedback
- Remove suggestion format guidance from report_progress (not applicable)
- De-duplicate description across Comment, EditComment, ReplyToReviewComment
- Drop outer fence in suggestion format example
- Clarify that suggestions only work for self-contained changes
- Remove useless example comment from review tool description
---------
Co-authored-by: pullfrog <team@pullfrog.com>
