* fix(log-audit): kill 404 noise from `/api/github/installation-token` at source (#693)
Closes#693. Issue diagnosed a surface symptom (`log.error` on expected
404s) but missed the actual root causes. Investigation revealed two
distinct populations producing identical 3-call 404 bursts:
1. **Fork-CI on `pullfrog/pullfrog`**: `test-token.yml` and
`trigger-sync.yml` ship with `on: push: main`, so every fork inherits
them and 404s our token endpoint on first push. Self-inflicted noise
that scales with fork count.
2. **Real users hitting the full action without installing the App**:
`/api/repo/.../run-context` uses the caller's `GITHUB_TOKEN` to read
the repo from GitHub and then unconditionally lazy-provisions
Account+Repo rows via `fetchOrCreateRepo`, even when the App isn't
installed. Generates phantom DB rows and false `new account created`
team@ alerts. (Confirmed via Prisma: `ezcorp-org` has an Account row
with `installerLogin: null`, never installed our App.)
Both populations then trip the client retry loop in
`acquireTokenViaOIDC`, which matched `"Token exchange failed"` and
retried 3× on terminal 4xx — tripling log volume and wasting CI time.
## Changes
- `action/.github/workflows/{test-token,trigger-sync}.yml`: gate jobs
with `if: github.repository == 'pullfrog/pullfrog'`. Forks inherit
the files but the jobs no-op.
- `app/api/repo/[owner]/[repo]/run-context/route.ts`: call
`getRepoInstallation` first; return 404 with install URL if the App
isn't installed, before any DB writes or GitHub repo fetch.
- `action/utils/github.ts`: introduce `TokenExchangeError` for non-2xx
server responses; `acquireNewToken` no longer retries it. Retry now
fires only on genuine network/timeout failures. 404 surfaces a
user-actionable error pointing at the install URL.
- `app/api/github/installation-token/route.ts`: move `log.error` inside
the 500 branch only. 404 branch is silent (expected user-state) and
returns the same install URL message for consistency.
## Effect
- Better Stack `level=error` lines from this path: 6/day → 0.
- Failed user-trial CI time: 3 wasted token requests → 1.
- User-facing error: opaque `Token exchange failed: 404` → actionable
install URL.
- No more phantom Account rows from never-installed callers.
Skipped per design discussion: phantom-account cleanup (conservative —
stop the bleed, leave history), `AGENTS.md` rule (overgeneralized).
* review: address oracle leak + per-env install URL + retryable 5xx
Addresses pullfrog[bot] (IMPORTANT) and Copilot review findings on #708:
- **Install-status oracle in `run-context`** [pullfrog, Copilot]:
`getRepoInstallation` runs with our App's JWT, *before* the caller's
bearer token is validated against the repo. Pre-PR the route was
uniformly bad-token-shaped; the new install-specific 404 turned it
into an unauthenticated oracle distinguishing "Pullfrog installed
here" from "not installed". Collapsed the 404 message to match the
outer catch's ambiguous "repository not found or token lacks access".
Legit runners still get the actionable install URL from
`/api/github/installation-token`, which IS gated by OIDC.
- **Hardcoded `github.com/apps/pullfrog`** [Copilot]: server-side
`installation-token` now uses `GITHUB_APP_INSTALL_URL` from
`app/globals.ts`, so dev/staging deployments with a different
`GITHUB_APP_SLUG` direct users to the correct app. Action-side
echoes the server's `error` body when present (single source of
truth) and falls back to a generic message only if the body isn't
JSON.
- **Transient 5xx/429 made terminal** [Copilot]: `shouldRetry` now
returns `true` for `TokenExchangeError` with `status >= 500` or
`status === 429`. 4xx remains terminal (the actual #693 fix). Real
outages no longer fail the workflow immediately.
- **Stale comment** [pullfrog, Copilot]: reworded the comment at
`installation-token/route.ts:141` to reflect the new retry policy
("the action surfaces this once (no retry)" instead of "the action
retries on this").
* review: restore caller-token-first auth in run-context
Pre-PR, `getEnrichedRepo({owner, repo, token})` used the caller's
token as the auth boundary — `getRepo({token})` succeeding was the
proof-of-access check. My initial install-gate inverted the order
and ran the App-credentialed `getRepoInstallation` first, which is
how it became:
- an install-status oracle (pullfrog bot, addressed previously by
matching the outer-catch wording), and
- an outbound amplifier against our App JWT for arbitrary `owner/repo`
(pullfrog bot, this commit).
Reordered so `getRepo({token})` runs first. Garbage / unauthorized
bearers get rejected by github (mapped to 403 by the outer catch)
before any App-credentialed call fires. `getRepo` is cached 5min,
so `getEnrichedRepo` below remains a free re-hit.
Pullfrog is a GitHub bot that brings the full power of your favorite coding agents into GitHub. It's open source and powered by GitHub Actions.
Tag @pullfrog — Tag @pullfrog in a comment anywhere in your repo. It will pull in any relevant context using the action's internal MCP server and perform the appropriate task.
Prompt from the web — Trigger arbitrary tasks from the Pullfrog dashboard
Automated triggers — Configure Pullfrog to trigger agent runs in response to specific events. Each of these triggers can be associated with custom prompt instructions.
issue created
issue labeled
PR created
PR review created
PR review requested
and more...
Pullfrog is the bridge between your preferred coding agents and GitHub. Use it for:
🤖 Coding tasks — Tell @pullfrog to implement something and it'll spin up a PR. If CI fails, it'll read the logs and attempt a fix automatically. It'll automatically address any PR reviews too.
🔍 PR review — Coding agents are great at reviewing PRs. Using the "PR created" trigger, you can configure Pullfrog to auto-review new PRs.
🤙 Issue management — Via the "issue created" trigger, Pullfrog can automatically respond to common questions, create implementation plans, and link to related issues/PRs. Or (if you're feeling lucky) you can prompt it to immediately attempt a PR addressing new issues.
Literally whatever — Want to have the agent automatically add docs to all new PRs? Cut a new release with agent-written notes on every commit to main? Pullfrog lets you do it.
Standalone Usage
You can also use pullfrog/pullfrog as a step in your own workflows. The action exposes a result output that can be consumed by subsequent steps.
Example: Auto-generate release notes on new tags
name:Releaseon:push:tags:['v*']permissions:contents:writejobs:release:runs-on:ubuntu-lateststeps:- name:Checkoutuses:actions/checkout@v4with:fetch-depth:0- name:Generate release notesid:notesuses:pullfrog/pullfrog@v0with:prompt:| Generate release notes for ${{ github.ref_name }}.
Compare commits between this tag and the previous tag.
Format as markdown: summary paragraph, then ### Features, ### Fixes, ### Breaking Changes sections.
Omit empty sections. Be concise.env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}# write to file to avoid shell escaping issues with special characters- name:Create GitHub releaserun:| notesfile="$RUNNER_TEMP/release-notes-$GITHUB_RUN_ID.md"
printf '%s' "$NOTES" > "$notesfile"
gh release create ${{ github.ref_name }} --title "${{ github.ref_name }}" --notes-file "$notesfile"env:GH_TOKEN:${{ github.token }}NOTES:${{ steps.notes.outputs.result }}
Example: Structured Output with Zod Schema
You can force the agent to return structured JSON output by providing a JSON schema. This allows you to reliably parse and use the agent's response in subsequent workflow steps.
You can define your JSON schema directly or uou can use any validation library that converts to JSON Schema. Here's an example using Zod:
name:Release Checkon:pull_request:types:[closed]jobs:check-release:if:github.event.pull_request.merged == trueruns-on:ubuntu-lateststeps:- uses:actions/checkout@v4- name:Install dependenciesrun:npm install --no-save --no-package-lock zod @actions/core- name:Generate Schemaid:schemarun:| node -e '
import { z } from "zod";
import { setOutput } from "@actions/core";
const schema = z.object({
version: z.string().describe("Semantic version number (e.g. 1.0.0)"),
isBreaking: z.boolean().describe("Whether this release contains breaking changes"),
changelog: z.array(z.string()).describe("List of changes in this release"),
});
setOutput("schema", JSON.stringify(z.toJSONSchema(schema)));
'- name:Analyze PRid:analysisuses:pullfrog/pullfrog@v0with:prompt:| Analyze this PR and determine semantic versioning impact.
Return a JSON object matching the provided schema.output_schema:${{ steps.schema.outputs.schema }}env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}- name:Process Resultrun:| # Parse the JSON result using fromJSON()
echo "Version: ${{ fromJSON(steps.analysis.outputs.result).version }}"
echo "Breaking: ${{ fromJSON(steps.analysis.outputs.result).isBreaking }}"