post-run gate: fail review-mode runs that don't submit a review or progress (#638)
* post-run gate: fail the run when review mode finishes without a review or progress
review-mode runs that ended in a text-only assistant turn ("now I have enough
to draft the review...") were silently swallowed: the progress comment was
deleted by stranded-comment cleanup and no review appeared on the PR. user-
visible result was identical to "the agent never ran." caught in
https://github.com/pullfrog/app/actions/runs/25583698781.
new post-run gate alongside stopHook / dirtyTree / summaryStale: derived
inline from toolState (selectedMode in {Review, IncrementalReview} && !review
&& !finalSummaryWritten && hadProgressComment) — no parallel toolState flag.
when it fires, the resume prompt nudges the agent to call either
create_pull_request_review or report_progress; persistent failure after
MAX_POST_RUN_RETRIES surfaces as AgentResult.error.
also: when the post-run loop returns success=false, write the error to the
progress comment before the stranded-comment cleanup runs, and skip the
delete in that case. previously a !success run from the loop would lose the
error message into the void.
IncrementalReview's trivial-skip branch now calls report_progress with a
brief "no review warranted" note instead of exiting silently — keeps the
contract symmetric with the gate and gives the user a visible signal even
on no-op review runs.
documents the literal-record design rule on the ToolState interface so
future fields don't drift back into derived/absence-encoding state.
* review feedback: mode-aware nudge, gate-error preservation, prompt order
addresses three findings from the auto-review on this PR:
1. Review mode nudge no longer offers `report_progress` as an exit. Review
mode's contract (modes.ts step 5) forbids it; the gate previously sent
contradictory copy. IncrementalReview's nudge still offers both since
its trivial-skip path legitimately allows `report_progress`.
2. `writeJobSummary` is now wrapped in try/catch on the success-path
cleanup. without this, a throw there jumped to the outer catch and
overwrote the gate's failure message in the progress comment with the
(less actionable) writeJobSummary error — restoring exactly the
invisible-failure UX this PR fixes. step-summary writes are
informational; let them fail silently.
3. `buildPostRunPrompt` reorders gates to match the terminal hard-fail
order: `stopHook` → `unsubmittedReview` → `dirtyTree` → `summaryStale`.
when both hard-fail gates co-fire (rare in review modes), the prompt's
emphasis now matches the user-visible failure message.
new test asserts the IncrementalReview nudge offers both exits while the
Review nudge offers only `create_pull_request_review`. e2e validation
already passed against pullfrog/preview-638-review-stop-hook PR #1
(gate fired once; agent recovered on second turn).
* mode-aware terminal error copy
second auto-review caught a residual contradiction: the terminal hard-fail
error string reported "create_pull_request_review or report_progress" for
both modes, even though the new mode-aware nudge tells Review-mode agents
"Review mode does not have a no-submit exit". the error message now mirrors
the nudge — Review names only `create_pull_request_review`,
IncrementalReview lists both. additional Review-mode hard-fail test asserts
the absence of `report_progress` in the error.
This commit is contained in:
committed by
pullfrog[bot]
parent
653fae47a5
commit
85d25a6fe6
@@ -68,6 +68,28 @@ export type StoredPushDest = {
|
||||
localBranch: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* mutable per-run record of facts that occurred during execution. shared
|
||||
* between the action process and the MCP server (one process — toolState is
|
||||
* just a JS object passed by reference into both surfaces).
|
||||
*
|
||||
* design rule: ToolState is LITERAL. each field records a thing that
|
||||
* happened — `review` is set when `create_pull_request_review` succeeded,
|
||||
* `finalSummaryWritten` flips when `report_progress` wrote a non-plan body,
|
||||
* `selectedMode` is set when `select_mode` was called. fields should never
|
||||
* encode the absence of an event ("unsubmittedReview", "missingArtifact"),
|
||||
* speculative state, or values derived from other fields.
|
||||
*
|
||||
* any predicate the rest of the code needs ("the agent picked review mode but
|
||||
* never produced a review or progress write") is computed inline at the call
|
||||
* site, not stored. derived state in this struct invariably drifts from the
|
||||
* literal fields under refactors and is the wrong layer for the check.
|
||||
*
|
||||
* write narrowly: prefer adding state inside the tool that mutates it (e.g.
|
||||
* `create_pull_request_review` populates `toolState.review`) and reading
|
||||
* narrowly elsewhere. don't introduce flags from main.ts that mirror what an
|
||||
* MCP tool already records.
|
||||
*/
|
||||
export interface ToolState {
|
||||
// where we're allowed to push - base repo initially, fork URL for fork PRs
|
||||
// set by setupGit, updated by checkout_pr. always set before push validation.
|
||||
|
||||
Reference in New Issue
Block a user