PR #790 (the first bot-authored models-bump PR) shipped a broken bump for
openrouter/gemini-flash: the bot pattern-matched the parallel google bump
and fabricated `openrouter/google/gemini-3.5-flash`, which exists on the
OpenRouter API but not on models.dev's openrouter section — the catalog
OpenCode actually reads. The slug failed at runtime with `Model not found`.
Two CI gaps let it through:
1. `models-live` matrix pruned every `openrouter/*` and keyed `opencode/*`
alias as a "passthrough", smoke-testing only one canary per routing
layer. But those aren't passthroughs — each is a distinct models.dev
catalog entry that can drift independently of the direct-provider
mirror. Drop the pruning; smoke every keyed alias (53 jobs, up from 25).
Only `bedrock/byok` stays pruned (sentinel resolve).
2. `models-catalog` test (the integrity gate that asserts every resolve
exists on models.dev) was main-only by design — to keep upstream catalog
churn from blocking unrelated PRs. But it's exactly the test we want
running on the bot's own catalog edits. Add `pullfrog/models-bump`
head-ref to its trigger.
Also tighten the bot prompt in models-bump.yml: new rule 0 requires every
new `resolve` to equal `<alias-provider>/<c.modelId>` for some `c` in the
alias's own `candidates[]` in models-bump-context.json — the deterministic
preprocessor only emits candidates sourced from models.dev's mirror, so
this gates against the cross-alias pattern-matching that broke PR #790.
For `openRouterResolve` the gate is `openRouterCandidates[]` (OpenRouter
API), which is necessary but not sufficient; the `models-catalog` job is
the authoritative models.dev check.
Verified locally:
- baseline `pnpm -C action test:catalog` passes 133 tests
- simulated the PR #790 hunk (sed'd `openrouter/google/gemini-3.5-flash`
into action/models.ts) and the catalog test fails with the right
assertion: `model "google/gemini-3.5-flash" not found under openrouter
on models.dev`
- `FULL=1 node action/test/matrix.ts` emits 53 aliases (was 25); every
openrouter/* alias and every keyed opencode/* alias now smoked
Pullfrog is a GitHub bot that brings the full power of your favorite coding agents into GitHub. It's open source and powered by GitHub Actions.
Tag @pullfrog — Tag @pullfrog in a comment anywhere in your repo. It will pull in any relevant context using the action's internal MCP server and perform the appropriate task.
Prompt from the web — Trigger arbitrary tasks from the Pullfrog dashboard
Automated triggers — Configure Pullfrog to trigger agent runs in response to specific events. Each of these triggers can be associated with custom prompt instructions.
issue created
issue labeled
PR created
PR review created
PR review requested
and more...
Pullfrog is the bridge between your preferred coding agents and GitHub. Use it for:
🤖 Coding tasks — Tell @pullfrog to implement something and it'll spin up a PR. If CI fails, it'll read the logs and attempt a fix automatically. It'll automatically address any PR reviews too.
🔍 PR review — Coding agents are great at reviewing PRs. Using the "PR created" trigger, you can configure Pullfrog to auto-review new PRs.
🤙 Issue management — Via the "issue created" trigger, Pullfrog can automatically respond to common questions, create implementation plans, and link to related issues/PRs. Or (if you're feeling lucky) you can prompt it to immediately attempt a PR addressing new issues.
Literally whatever — Want to have the agent automatically add docs to all new PRs? Cut a new release with agent-written notes on every commit to main? Pullfrog lets you do it.
Standalone Usage
You can also use pullfrog/pullfrog as a step in your own workflows. The action exposes a result output that can be consumed by subsequent steps.
Example: Auto-generate release notes on new tags
name:Releaseon:push:tags:['v*']permissions:contents:writejobs:release:runs-on:ubuntu-lateststeps:- name:Checkoutuses:actions/checkout@v4with:fetch-depth:0- name:Generate release notesid:notesuses:pullfrog/pullfrog@v0with:prompt:| Generate release notes for ${{ github.ref_name }}.
Compare commits between this tag and the previous tag.
Format as markdown: summary paragraph, then ### Features, ### Fixes, ### Breaking Changes sections.
Omit empty sections. Be concise.env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}# write to file to avoid shell escaping issues with special characters- name:Create GitHub releaserun:| notesfile="$RUNNER_TEMP/release-notes-$GITHUB_RUN_ID.md"
printf '%s' "$NOTES" > "$notesfile"
gh release create ${{ github.ref_name }} --title "${{ github.ref_name }}" --notes-file "$notesfile"env:GH_TOKEN:${{ github.token }}NOTES:${{ steps.notes.outputs.result }}
Example: Structured Output with Zod Schema
You can force the agent to return structured JSON output by providing a JSON schema. This allows you to reliably parse and use the agent's response in subsequent workflow steps.
You can define your JSON schema directly or uou can use any validation library that converts to JSON Schema. Here's an example using Zod:
name:Release Checkon:pull_request:types:[closed]jobs:check-release:if:github.event.pull_request.merged == trueruns-on:ubuntu-lateststeps:- uses:actions/checkout@v4- name:Install dependenciesrun:npm install --no-save --no-package-lock zod @actions/core- name:Generate Schemaid:schemarun:| node -e '
import { z } from "zod";
import { setOutput } from "@actions/core";
const schema = z.object({
version: z.string().describe("Semantic version number (e.g. 1.0.0)"),
isBreaking: z.boolean().describe("Whether this release contains breaking changes"),
changelog: z.array(z.string()).describe("List of changes in this release"),
});
setOutput("schema", JSON.stringify(z.toJSONSchema(schema)));
'- name:Analyze PRid:analysisuses:pullfrog/pullfrog@v0with:prompt:| Analyze this PR and determine semantic versioning impact.
Return a JSON object matching the provided schema.output_schema:${{ steps.schema.outputs.schema }}env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}- name:Process Resultrun:| # Parse the JSON result using fromJSON()
echo "Version: ${{ fromJSON(steps.analysis.outputs.result).version }}"
echo "Breaking: ${{ fromJSON(steps.analysis.outputs.result).isBreaking }}"