David Blass ada5584737 test(mcp): make checkout/reviewComments tests offline (fixture-driven) (#575)
`mcp/checkout.test.ts` and `mcp/reviewComments.test.ts` previously hit
live GitHub on every run via `acquireNewToken()`, requiring `GH_TOKEN`
or `GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` in the env. that made them:

- cred-gated — the action runtime filters `_KEY$` / `_TOKEN$` from
  subprocess env, so the husky pre-push hook (which runs
  `pnpm -r test`) blocked Pullfrog agents from pushing branches. issues
  #562, #563, #564, #566 all hit this exact blocker and never got their
  fixes pushed.
- non-deterministic and slow (network round-trips for a snapshot test).

both tests are really snapshot tests of pure formatters
(`formatFilesWithLineNumbers`, plus `parseFilePatches` /
`buildThreadBlocks` / `formatReviewThreads` for review data). the live
fetches were just an inefficient way to obtain fixtures.

changes:

1. extract a pure `formatReviewData({ review, threads, prFiles, ... })`
   from `getReviewData` in `mcp/reviewComments.ts`. `getReviewData`
   becomes thin orchestration: fetch + call formatter. preserves the
   "skip listFiles when no threads" perf optimization.

2. add `action/mcp/__fixtures__/` with checked-in JSON captures for the
   three fixture test cases (pullfrog/test-repo#1 listFiles,
   pullfrog/scratch#49 review 3485940013, pullfrog/scratch#64 review
   3531000326). ~14KB total. fixtures store only the fields the
   formatter reads — volatile fields (sha, blob_url, etc.) are dropped.

3. rewrite both test files to load the fixtures and call the pure
   formatters directly. snapshot keys updated; snapshot content
   unchanged (verified by running existing snapshots against the
   refactored tests).

4. add `action/scripts/refresh-test-fixtures.ts` to re-fetch the
   fixtures from live GitHub on demand:
   `node action/scripts/refresh-test-fixtures.ts` (with creds in
   `.env` or env). re-run when the GitHub API response shape changes
   and review the snapshot diff.

trade-off: a silent change to GitHub's `pulls.listFiles` /
`pulls.getReview` / GraphQL `reviewThreads` response shape would no
longer break this test on every push. that tradeoff is worth it: shape
drift on those endpoints is rare (years between changes), and a
dedicated cron that runs the refresh script and opens a PR on diff is
a far better signal than a flaky cred-gated pre-push hook.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-05 23:25:46 +00:00
2026-01-16 08:00:16 +00:00
2026-03-12 05:22:51 +00:00
2025-08-27 16:53:48 -07:00
2026-01-19 08:41:56 +00:00
2026-05-05 05:16:59 +00:00
2026-05-05 17:12:36 +00:00
2026-03-12 05:22:51 +00:00
2026-03-12 05:22:51 +00:00

Green Pullfrog logo
Pullfrog

Bring your favorite coding agent into GitHub


🚀 Pullfrog is in beta! We're onboarding users in waves. Get on the waitlist →


What is Pullfrog?

Pullfrog is a GitHub bot that brings the full power of your favorite coding agents into GitHub. It's open source and powered by GitHub Actions.

  • Tag @pullfrog — Tag @pullfrog in a comment anywhere in your repo. It will pull in any relevant context using the action's internal MCP server and perform the appropriate task.
  • Prompt from the web — Trigger arbitrary tasks from the Pullfrog dashboard
  • Automated triggers — Configure Pullfrog to trigger agent runs in response to specific events. Each of these triggers can be associated with custom prompt instructions.
    • issue created
    • issue labeled
    • PR created
    • PR review created
    • PR review requested
    • and more...

Pullfrog is the bridge between your preferred coding agents and GitHub. Use it for:

  • 🤖 Coding tasks — Tell @pullfrog to implement something and it'll spin up a PR. If CI fails, it'll read the logs and attempt a fix automatically. It'll automatically address any PR reviews too.
  • 🔍 PR review — Coding agents are great at reviewing PRs. Using the "PR created" trigger, you can configure Pullfrog to auto-review new PRs.
  • 🤙 Issue management — Via the "issue created" trigger, Pullfrog can automatically respond to common questions, create implementation plans, and link to related issues/PRs. Or (if you're feeling lucky) you can prompt it to immediately attempt a PR addressing new issues.
  • Literally whatever — Want to have the agent automatically add docs to all new PRs? Cut a new release with agent-written notes on every commit to main? Pullfrog lets you do it.

Standalone Usage

You can also use pullfrog/pullfrog as a step in your own workflows. The action exposes a result output that can be consumed by subsequent steps.

Example: Auto-generate release notes on new tags

name: Release
on:
  push:
    tags: ['v*']

permissions:
  contents: write

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate release notes
        id: notes
        uses: pullfrog/pullfrog@v0
        with:
          prompt: |
            Generate release notes for ${{ github.ref_name }}.
            Compare commits between this tag and the previous tag.
            Format as markdown: summary paragraph, then ### Features, ### Fixes, ### Breaking Changes sections.
            Omit empty sections. Be concise.
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      # write to file to avoid shell escaping issues with special characters
      - name: Create GitHub release
        run: |
          notesfile="$RUNNER_TEMP/release-notes-$GITHUB_RUN_ID.md"
          printf '%s' "$NOTES" > "$notesfile"
          gh release create ${{ github.ref_name }} --title "${{ github.ref_name }}" --notes-file "$notesfile"
        env:
          GH_TOKEN: ${{ github.token }}
          NOTES: ${{ steps.notes.outputs.result }}

Example: Structured Output with Zod Schema

You can force the agent to return structured JSON output by providing a JSON schema. This allows you to reliably parse and use the agent's response in subsequent workflow steps.

You can define your JSON schema directly or uou can use any validation library that converts to JSON Schema. Here's an example using Zod:

name: Release Check
on:
  pull_request:
    types: [closed]

jobs:
  check-release:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: npm install --no-save --no-package-lock zod @actions/core

      - name: Generate Schema
        id: schema
        run: |
          node -e '
            import { z } from "zod";
            import { setOutput } from "@actions/core";
            const schema = z.object({
              version: z.string().describe("Semantic version number (e.g. 1.0.0)"),
              isBreaking: z.boolean().describe("Whether this release contains breaking changes"),
              changelog: z.array(z.string()).describe("List of changes in this release"),
            });
            setOutput("schema", JSON.stringify(z.toJSONSchema(schema)));
          '

      - name: Analyze PR
        id: analysis
        uses: pullfrog/pullfrog@v0
        with:
          prompt: |
            Analyze this PR and determine semantic versioning impact.
            Return a JSON object matching the provided schema.
          output_schema: ${{ steps.schema.outputs.schema }}
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      - name: Process Result
        run: |
          # Parse the JSON result using fromJSON()
          echo "Version: ${{ fromJSON(steps.analysis.outputs.result).version }}"
          echo "Breaking: ${{ fromJSON(steps.analysis.outputs.result).isBreaking }}"
S
Description
Self-hosted Ollama-powered code review bot for Gitea Actions based on pullfrog
Readme MIT 27 MiB
Languages
TypeScript 97.9%
Shell 0.7%
Dockerfile 0.7%
JavaScript 0.7%