ShockVPN/shockbot
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(mcp): make checkout/reviewComments tests offline (fixture-driven) (#575)

Browse Source 
`mcp/checkout.test.ts` and `mcp/reviewComments.test.ts` previously hit
live GitHub on every run via `acquireNewToken()`, requiring `GH_TOKEN`
or `GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` in the env. that made them:

- cred-gated — the action runtime filters `_KEY$` / `_TOKEN$` from
  subprocess env, so the husky pre-push hook (which runs
  `pnpm -r test`) blocked Pullfrog agents from pushing branches. issues
  #562, #563, #564, #566 all hit this exact blocker and never got their
  fixes pushed.
- non-deterministic and slow (network round-trips for a snapshot test).

both tests are really snapshot tests of pure formatters
(`formatFilesWithLineNumbers`, plus `parseFilePatches` /
`buildThreadBlocks` / `formatReviewThreads` for review data). the live
fetches were just an inefficient way to obtain fixtures.

changes:

1. extract a pure `formatReviewData({ review, threads, prFiles, ... })`
   from `getReviewData` in `mcp/reviewComments.ts`. `getReviewData`
   becomes thin orchestration: fetch + call formatter. preserves the
   "skip listFiles when no threads" perf optimization.

2. add `action/mcp/__fixtures__/` with checked-in JSON captures for the
   three fixture test cases (pullfrog/test-repo#1 listFiles,
   pullfrog/scratch#49 review 3485940013, pullfrog/scratch#64 review
   3531000326). ~14KB total. fixtures store only the fields the
   formatter reads — volatile fields (sha, blob_url, etc.) are dropped.

3. rewrite both test files to load the fixtures and call the pure
   formatters directly. snapshot keys updated; snapshot content
   unchanged (verified by running existing snapshots against the
   refactored tests).

4. add `action/scripts/refresh-test-fixtures.ts` to re-fetch the
   fixtures from live GitHub on demand:
   `node action/scripts/refresh-test-fixtures.ts` (with creds in
   `.env` or env). re-run when the GitHub API response shape changes
   and review the snapshot diff.

trade-off: a silent change to GitHub's `pulls.listFiles` /
`pulls.getReview` / GraphQL `reviewThreads` response shape would no
longer break this test on every push. that tradeoff is worth it: shape
drift on those endpoints is rare (years between changes), and a
dedicated cron that runs the refresh script and opens a PR on diff is
a far better signal than a flaky cred-gated pre-push hook.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
David Blass
2026-05-05 23:25:46 +00:00
committed by pullfrog[bot]
parent b6e2c61d30
commit ada5584737
9 changed files with 506 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
mcp/__snapshots__/checkout.test.ts.snap
+2 -2
View File
@@ -1,6 +1,6 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`fetchAndFormatPrDiff > generates accurate TOC line numbers for pullfrog/test-repo#1 > content 1`] = `
exports[`formatFilesWithLineNumbers > generates accurate TOC line numbers for pullfrog/test-repo#1 > content 1`] = `
"## Files (5)
- src/format.ts → lines 9-32 · diff-41c7b3ac268a3a1ae5c7be92f1230f600013b7170e44a693570ccbdb183ea36b
- src/math.ts → lines 33-55 · diff-9c6e445a719b33e276684bdf95c69e617f0303638d44cf90d61295f2720ecc63
@@ -96,7 +96,7 @@ diff --git a/test/math.test.ts b/test/math.test.ts
"
`;
exports[`fetchAndFormatPrDiff > generates accurate TOC line numbers for pullfrog/test-repo#1 > toc 1`] = `
exports[`formatFilesWithLineNumbers > generates accurate TOC line numbers for pullfrog/test-repo#1 > toc 1`] = `
"## Files (5)
- src/format.ts → lines 9-32 · diff-41c7b3ac268a3a1ae5c7be92f1230f600013b7170e44a693570ccbdb183ea36b
- src/math.ts → lines 33-55 · diff-9c6e445a719b33e276684bdf95c69e617f0303638d44cf90d61295f2720ecc63
mcp/__snapshots__/reviewComments.test.ts.snap
+4 -4
View File
@@ -1,6 +1,6 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`getFormattedReviewThreads > formats body-only review > content 1`] = `
exports[`formatReviewData > formats body-only review > content 1`] = `
"# Review Threads (0) for PR #64 - Review 3531000326 by pullfrog[bot]
## Review Body
@@ -11,9 +11,9 @@ This PR looks great. The retry logic is well-implemented and the tests are compr
"
`;
exports[`getFormattedReviewThreads > formats body-only review > toc 1`] = `""`;
exports[`formatReviewData > formats body-only review > toc 1`] = `""`;
exports[`getFormattedReviewThreads > formats thread blocks with TOC and correct line numbers > content 1`] = `
exports[`formatReviewData > formats thread blocks with TOC and correct line numbers > content 1`] = `
"# Review Threads (1) for PR #49 - Review 3485940013 by cursor[bot]
## TOC
@@ -68,4 +68,4 @@ LOCATIONS END -->
"
`;
exports[`getFormattedReviewThreads > formats thread blocks with TOC and correct line numbers > toc 1`] = `"- .github/workflows/test.yml:7 → lines 25-52"`;
exports[`formatReviewData > formats thread blocks with TOC and correct line numbers > toc 1`] = `"- .github/workflows/test.yml:7 → lines 25-52"`;