add diff coverage preflight for PR review submissions (#544)
* add one-time diff coverage preflight for PR reviews track diff read coverage from agent tool-use events and run a one-time pre-flight before review submission, with explicit coverage skip reasons for low-value files like lockfiles. Made-with: Cursor * add manual dispatch fallback for preview deploy workflow allow preview repo and preview sync jobs to be run via workflow_dispatch with explicit PR number and branch inputs, so preview provisioning can be retriggered when pull_request events fail to fire. Made-with: Cursor * fix manual preview dispatch PR input wiring use normalized PR number and branch env values for comment creation and script env wiring so workflow_dispatch preview runs can create and update PR-specific preview resources. Made-with: Cursor * remove obsolete snapshots invalidated by checkout instructions change * fix diff coverage read offset handling and add local sanity-check guidance normalize read offset semantics for diff coverage tracking, reuse shared range counting in review preflight, add focused diff coverage unit tests, and document the local play.ts testing workflow in AGENTS.md. Made-with: Cursor * add regenerated mcp test snapshots capture snapshot files generated by the review comment and checkout formatting tests during pre-push validation so the branch remains clean and reproducible. Made-with: Cursor * add diff coverage preflight instrumentation logs log diff coverage initialization in checkout_pr and emit preflight state/breakdown diagnostics in create_pull_request_review to debug missing coverage enforcement in preview e2e runs. Made-with: Cursor * add env override to force local cli execution in action runtime support explicit local-cli execution via PULLFROG_FORCE_LOCAL_CLI so preview workflows can run branch action code instead of the npm fallback package during e2e debugging. Made-with: Cursor * add preview e2e debugging learnings for action runtime validation capture the preview execution-path gotchas and one-time preflight verification pattern in AGENTS.md so future investigations validate the real runtime and avoid npm fallback confusion. Made-with: Cursor * reduce diff coverage log noise while preserving failure visibility downgrade verbose diff coverage lifecycle diagnostics to debug, keep a concise info-level pre-flight failure signal, and document preview runtime debugging learnings in AGENTS.md. Made-with: Cursor * WIP * tune sync.md: ff override + softer overlap verification Made-with: Cursor * chore: bump models snapshot for claude-opus-4-7 Made-with: Cursor * rip out coverage_skips waiver from diff coverage pre-flight Made-with: Cursor --------- Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
This commit is contained in:
committed by
pullfrog[bot]
parent
5e6ff67623
commit
56a5d29598
@@ -130,7 +130,7 @@ ${learningsStep(t, 6)}`,
|
||||
"Review code, PRs, or implementations; provide feedback or suggestions; identify issues; or check code quality, style, and correctness",
|
||||
prompt: `### Checklist
|
||||
|
||||
1. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata and a \`diffPath\`. Read the diff to identify the major areas of change.
|
||||
1. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata and a \`diffPath\`. read the diff TOC first and treat its file line ranges as your coverage checklist.
|
||||
|
||||
2. For each area of change:
|
||||
- read the diff and trace data flow, check boundaries, and verify assumptions
|
||||
@@ -147,6 +147,7 @@ ${learningsStep(t, 6)}`,
|
||||
4. Submit — ALWAYS submit exactly one review via \`${t("create_pull_request_review")}\`.
|
||||
Do NOT call \`report_progress\` — the review is the final record and the progress
|
||||
comment will be cleaned up automatically.
|
||||
note: the first create_pull_request_review submission may error with a one-time diff-coverage nudge listing unread TOC regions. retry the same call to proceed — optionally after reading the listed ranges. the pre-flight will not block again this session.
|
||||
|
||||
- **critical issues** (blocks merge — bugs, security, data loss):
|
||||
\`approved: false\`. Body begins with a GitHub alert blockquote, e.g.:
|
||||
@@ -165,7 +166,7 @@ ${learningsStep(t, 6)}`,
|
||||
"Re-review a PR after new commits are pushed; focus on new changes since the last review",
|
||||
prompt: `### Checklist
|
||||
|
||||
1. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata, \`diffPath\` (full diff), and \`incrementalDiffPath\` (changes since last reviewed version, if available).
|
||||
1. Checkout the PR via \`${t("checkout_pr")}\` — this returns PR metadata, \`diffPath\` (full diff), and \`incrementalDiffPath\` (changes since last reviewed version, if available). read the diff TOC first and use its line ranges as your coverage checklist.
|
||||
|
||||
2. If \`incrementalDiffPath\` is present, read it to see what changed since the last review. This is a range-diff that isolates the net changes, filtering out base branch noise. If not present, fall back to reviewing the full PR diff.
|
||||
|
||||
@@ -189,6 +190,7 @@ ${learningsStep(t, 6)}`,
|
||||
- in some cases you may receive a complete diff for the whole pull request instead of an incremental one. when this happens, you will need to determine what changes have happened since Pullfrog's most recent review.
|
||||
|
||||
7. Submit — Do NOT call \`report_progress\` or \`create_issue_comment\` — the review is the final record and the progress comment will be cleaned up automatically. the review body always includes the reviewed changes from step 6a. append \`Prior review feedback:\\n\` with the checklist from step 6b only if any prior comments were addressed. Follow these rules:
|
||||
- note: the first create_pull_request_review submission may error with a one-time diff-coverage nudge listing unread TOC regions. retry the same call to proceed — optionally after reading the listed ranges. the pre-flight will not block again this session.
|
||||
- IF NO NEW ISSUES, NON-SUBSTANTIVE CHANGES ONLY (trivial formatting, import reordering, comment tweaks): do NOT submit a review. Do NOT call \`report_progress\`. Exit — the progress comment will be cleaned up automatically.
|
||||
- ELSE IF NEW CRITICAL ISSUES (blocks merge): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with a GitHub alert blockquote (e.g. \`> [!CAUTION]\\n> This PR introduces ...\`), then the reviewed changes summary and prior feedback (if any).
|
||||
- ELSE IF NEW RECOMMENDED CHANGES (non-critical): call \`${t("create_pull_request_review")}\` with \`approved: false\`, all comments, and the review body. body opens with \`> [!IMPORTANT]\\n> ...\` alert, then the reviewed changes summary and prior feedback (if any).
|
||||
|
||||
Reference in New Issue
Block a user