live todo tracking, collapsible task list in final progress, hide set_output outside standalone (#492)

* fix false "without reporting progress" error + live todo tracking

clean up orphaned progress comments when review is skipped or only
set_output is used, preventing the false positive in handleAgentResult.

parse todowrite events from OpenCode's NDJSON stream and render a
live markdown checklist in the PR progress comment (2s debounce).
agent's explicit report_progress always takes priority.

Made-with: Cursor

* fix contradictory review/progress prompting

align Review and IncrementalReview mode prompts with their guidance —
mode prompts said "always submit" while guidance said "skip if clean."
remove the empty-approval submission that was silently dropped by the
tool. make progress comment lifecycle explicit: created on first call,
updated in place, removed after review submission.

Made-with: Cursor

* centralize todo tracking into shared TodoTracker module

extract inline todo tracking logic (~95 lines) from opentoad.ts into
action/utils/todoTracking.ts. the tracker is created once in main.ts
and passed to agents via AgentRunContext.todoTracker, making it
agent-agnostic and reusable for future agent implementations.

Made-with: Cursor

* fix todoTracker optional type to match file convention

add | undefined to todoTracker in AgentRunContext, matching every
other optional property in the same interface.

Made-with: Cursor

* instruct agents to always maintain a task list for live progress

system prompt now tells agents to create an internal task list at
the start of every run. the tracker renders it to the progress
comment automatically. report_progress is reserved for final
results only — no more intermediate "Checking..." messages that
cancel the tracker and leave stale text on the comment.

Made-with: Cursor

* require report_progress summary at end of every run

agents must always call report_progress with a final summary —
the completed task list should never be the end state of the
progress comment. updated all review mode prompts to call
report_progress after submitting (or not submitting) a review.

Made-with: Cursor

* keep progress comment after review with final summary

stop deleting the progress comment after review submission —
the agent now always calls report_progress with a summary at
the end, and that summary should persist as a record of what
was done.

Made-with: Cursor

* harden stranded progress comment cleanup

- main.ts: detect when tracker was last writer (agent never called
  report_progress) and delete the stranded checklist instead of
  leaving it as the final comment state
- postCleanup.ts: expand stuck-comment detection to also catch
  stranded todo checklists (regex match for checklist patterns)
  when the process is killed before normal cleanup runs
- modes.ts + selectMode.ts: add report_progress step to Summarize
  and SummaryUpdate modes (only modes that were missing it)

Made-with: Cursor

* fix stale comments, typo, and build mode redundancy

- comment.ts: update deleteProgressComment docstring and inline comment
  to reflect current usage (stranded-comment cleanup, not post-review)
- modes.ts: merge duplicate report_progress steps (8 + 10) into single
  step 9, fix "optimizatfixons" typo
- wiki/post-cleanup.md: document checklist detection regex

Made-with: Cursor

* collapsible completed todos in final progress, hide set_output outside standalone mode

- add renderCollapsible() to TodoTracker, append completed task list as
  <details> section when agent calls report_progress
- cancel tracker after agent's final report_progress so it doesn't
  overwrite with raw checklist
- conditionally register SetOutputTool only in standalone mode or when
  output_schema is provided
- remove unconditional set_output instruction from orchestrator task section
- update Summarize/SummaryUpdate mode guidance to not reference set_output

Made-with: Cursor

* show completion count in collapsible task list summary

Made-with: Cursor

* only count completed (not cancelled) in collapsible task list summary

Made-with: Cursor

* reinforce concise summary prompting across system prompt, modes, and tool description

Made-with: Cursor

* address review feedback: wasUpdated bypass, tracker false-positive, race condition

- remove wasUpdated=true from cleanup paths so handleAgentResult correctly
  detects genuinely silent runs
- add hadProgressComment to ToolState as immutable snapshot for the safety check
- use todoTracker.hasPublished instead of enabled for stranded-comment cleanup
- serialize onUpdate calls via inflightPromise chain with post-cancel guard
- add settled() to wait for in-flight updates before writing final summary

Made-with: Cursor

* address round-2 review: hasPublished after success, finalSummaryWritten flag

- set hasPublished only after onUpdate resolves (not before) so failed
  writes are not counted as published
- add finalSummaryWritten flag to ToolState, set after successful
  non-plan reportProgress; decouple cleanup detection from
  todoTracker.enabled so it survives API failures where cancel() ran
  but the write didn't succeed

Made-with: Cursor

mcp/comment.ts

@@ -329,7 +329,7 @@ export async function reportProgress(
     };
   }
 
-  // null = progress comment was deliberately deleted (e.g. by create_pull_request_review)
+  // null = progress comment was deleted by stranded-comment cleanup in main.ts
   if (existingCommentId === null) {
     return { body, action: "skipped" };
   }
@@ -400,17 +400,33 @@ export function ReportProgressTool(ctx: ToolContext) {
   return tool({
     name: "report_progress",
     description:
-      "Share progress on the associated GitHub issue/PR. Call this to post updates as you work. The first call creates a comment, subsequent calls update it. Use this throughout your work to keep stakeholders informed.",
+      "Share progress on the associated GitHub issue/PR. The first call creates a comment; subsequent calls update it in place. You MUST call this at the end of every run with a brief final summary (1-3 sentences). The completed task list is automatically appended in a collapsible section — do not restate individual steps.",
     parameters: ReportProgress,
     execute: execute(async (params) => {
-      const reportParams: { body: string; target_plan_comment?: boolean } = { body: params.body };
+      let body = params.body;
+
+      // for non-plan calls: stop auto-updates, wait for in-flight writes to settle,
+      // then append completed task list collapsible
+      if (!params.target_plan_comment && ctx.toolState.todoTracker) {
+        ctx.toolState.todoTracker.cancel();
+        await ctx.toolState.todoTracker.settled();
+        const collapsible = ctx.toolState.todoTracker.renderCollapsible();
+        if (collapsible) {
+          body = `${body}\n\n${collapsible}`;
+        }
+      }
+
+      const reportParams: { body: string; target_plan_comment?: boolean } = { body };
       if (params.target_plan_comment !== undefined) {
         reportParams.target_plan_comment = params.target_plan_comment;
       }
       const result = await reportProgress(ctx, reportParams);
 
+      if (!params.target_plan_comment) {
+        ctx.toolState.finalSummaryWritten = true;
+      }
+
       if (result.action === "skipped") {
-        // no-op: no comment target, but progress is still tracked for job summary
         return {
           success: true,
           message:
@@ -428,9 +444,9 @@ export function ReportProgressTool(ctx: ToolContext) {
 
 /**
  * Delete the progress comment if it exists.
- * Used after submitting a PR review since the review body contains all necessary info.
- * Sets progressCommentId to null, which prevents future report_progress calls from
- * creating a new comment (the agent may call report_progress again after this).
+ * Used by main.ts for stranded-comment cleanup (orphaned "Leaping into action" or
+ * checklist left by the todo tracker when the agent didn't call report_progress).
+ * Sets progressCommentId to null so subsequent report_progress calls are no-ops.
  */
 export async function deleteProgressComment(ctx: ToolContext): Promise<boolean> {
   const existingCommentId = ctx.toolState.progressCommentId;
@@ -455,7 +471,6 @@ export async function deleteProgressComment(ctx: ToolContext): Promise<boolean>
 
   // set to null (not undefined) so report_progress skips instead of creating a new comment
   ctx.toolState.progressCommentId = null;
-  ctx.toolState.wasUpdated = true;
 
   return true;
 }

mcp/review.ts

@@ -70,6 +70,7 @@ export function CreatePullRequestReviewTool(ctx: ToolContext) {
     description:
       "Submit a review for an existing pull request. " +
       "Each call creates a permanent, visible review on the PR — NEVER submit test or diagnostic reviews. " +
+      "Reviews with no body AND no comments are silently skipped (nothing to post). " +
       "IMPORTANT: 95%+ of feedback should be in 'comments' array with file paths and line numbers. " +
       "Only use 'body' for a 1-2 sentence summary with urgency and critical callouts. " +
       "Use 'suggestion' to propose replacement code - MUST preserve exact indentation of original code. " +

mcp/selectMode.ts

@@ -111,10 +111,9 @@ ${learningsStep(6)}`,
 
 3. Self-critique: review all drafted comments and drop any that are praise, style preferences, speculative/unverified claims, about pre-existing code unrelated to the PR, or not actionable.
 
-4. Submit a **single** review:
-   - call \`${ghPullfrogMcpName}/create_pull_request_review\` with all comments and a unified summary body
-   - call \`${ghPullfrogMcpName}/report_progress\` with the summary
-   - if no actionable issues found, skip the review — just call \`report_progress\` noting the PR was reviewed`,
+4. Submit:
+   - **actionable issues found**: call \`${ghPullfrogMcpName}/create_pull_request_review\` with all comments, a 1-3 sentence summary body, and \`approved: false\`. Then call \`report_progress\` with a 1-sentence summary.
+   - **no actionable issues found**: do NOT submit a review. Call \`${ghPullfrogMcpName}/report_progress\` with a brief note (e.g., "Reviewed — no issues found.").`,
 
   IncrementalReview: `### Checklist
 
@@ -134,10 +133,9 @@ ${learningsStep(6)}`,
 
 5. Self-critique: drop any comments that are praise, style preferences, speculative, about pre-existing code, or not actionable.
 
-6. Submit a **single** review:
-   - if actionable issues found: call \`${ghPullfrogMcpName}/create_pull_request_review\` with \`approved: false\`, all comments, and an **empty body** (do NOT include a summary — inline comments speak for themselves and a top-level comment clutters the PR conversation on every re-review)
-   - if no actionable issues found: submit with \`approved: true\` and an **empty body** (no inline comments, no summary)
-   - do NOT call \`${ghPullfrogMcpName}/report_progress\` — incremental reviews should be silent`,
+6. Submit:
+   - **actionable issues found**: call \`${ghPullfrogMcpName}/create_pull_request_review\` with \`approved: false\`, all comments, and an **empty body** — inline comments speak for themselves, and a top-level body clutters the PR conversation on every re-review cycle. Then call \`report_progress\` with a 1-sentence summary.
+   - **no actionable issues found**: do NOT submit a review. Call \`${ghPullfrogMcpName}/report_progress\` with a brief note (e.g., "Re-reviewed — no new issues found.").`,
 
   Plan: `### Checklist
 
@@ -209,8 +207,9 @@ ${learningsStep(4)}`,
    - PR metadata (title, file count, commit count, base/head branches)
    - format instructions from EVENT INSTRUCTIONS (if any); otherwise use default format: TL;DR, key changes list, per-change sections with plain-language \`##\` titles and before/after framing
    - instruct it to use the TOC to selectively read relevant diff sections, not the entire file
-   - instruct it to return the full summary markdown via \`${ghPullfrogMcpName}/set_output\`
+   - instruct it to return the full summary markdown as its final response
 3. After the subagent completes, call \`${ghPullfrogMcpName}/create_issue_comment\` with \`type: "Summary"\` and the summary body.
+4. Call \`${ghPullfrogMcpName}/report_progress\` with a brief note (e.g., "Posted PR summary.").
 
 ### Effort
 
@@ -226,8 +225,9 @@ An existing summary comment was found for this PR. Update it rather than creatin
    - the diff file path and PR metadata
    - the existing summary body (\`previousSummaryBody\`) so it can update rather than rewrite from scratch
    - format instructions from EVENT INSTRUCTIONS (if any)
-   - instruct it to produce an updated summary reflecting the current state of the PR and return via \`${ghPullfrogMcpName}/set_output\`
+   - instruct it to produce an updated summary reflecting the current state of the PR and return it as its final response
 4. After the subagent completes, call \`${ghPullfrogMcpName}/edit_issue_comment\` with \`commentId: existingSummaryCommentId\` (from this response) and the updated summary body.
+5. Call \`${ghPullfrogMcpName}/report_progress\` with a brief note (e.g., "Updated PR summary.").
 
 ### Effort
 

mcp/server.ts

@@ -10,6 +10,7 @@ import { log } from "../utils/cli.ts";
 import type { OctokitWithPlugins } from "../utils/github.ts";
 import type { ResolvedPayload } from "../utils/payload.ts";
 import type { RunContextData } from "../utils/runContextData.ts";
+import type { TodoTracker } from "../utils/todoTracking.ts";
 import { CheckoutPrTool } from "./checkout.ts";
 import { GetCheckSuiteLogsTool } from "./checkSuite.ts";
 import {
@@ -81,8 +82,14 @@ export interface ToolState {
   };
   // undefined = no comment yet, number = active comment, null = deliberately deleted
   progressCommentId: number | null | undefined;
+  // immutable snapshot: true if a progress comment was pre-created at init time.
+  // survives deleteProgressComment so handleAgentResult can still detect "expected but never reported".
+  hadProgressComment: boolean;
   lastProgressBody?: string;
   wasUpdated?: boolean;
+  // set after a non-plan report_progress successfully writes the final summary.
+  // decoupled from todoTracker.enabled so cleanup detection survives API failures.
+  finalSummaryWritten?: boolean;
   // set by select_mode when Plan + issue_number and plan-comment API returns existing plan (for report_progress target_plan_comment)
   existingPlanCommentId?: number;
   previousPlanBody?: string;
@@ -91,6 +98,7 @@ export interface ToolState {
   output?: string;
   usageEntries: AgentUsage[];
   model?: string | undefined;
+  todoTracker?: TodoTracker | undefined;
 }
 
 interface InitToolStateParams {
@@ -107,6 +115,7 @@ export function initToolState(params: InitToolStateParams): ToolState {
 
   return {
     progressCommentId: resolvedId,
+    hadProgressComment: !!resolvedId,
     backgroundProcesses: new Map(),
     usageEntries: [],
   };
@@ -192,9 +201,13 @@ function buildCommonTools(ctx: ToolContext, outputSchema?: JsonSchema): Tool<any
     GitTool(ctx),
     GitFetchTool(ctx),
     UploadFileTool(ctx),
-    SetOutputTool(ctx, outputSchema),
   ];
 
+  const isStandalone = ctx.payload.event.trigger === "unknown";
+  if (isStandalone || outputSchema) {
+    tools.push(SetOutputTool(ctx, outputSchema));
+  }
+
   // MCP shell with filtered env (no secrets leaked to child processes)
   if (ctx.payload.shell === "restricted") {
     tools.push(ShellTool(ctx));