Files
shockbot/utils/github.ts
T
Colin McDonnell 88f170e19a fix: 7 log-audit / run-audit findings (mega-PR) (#769)
* fix(#765): silence Clerk 400 (revoked OAuth) noise from getTokenForClerkId

Branch on isClerkAPIResponseError + status<500 so the well-understood
revoked-token redirect doesn't emit a level=error line in Better Stack
on every request. Vercel maps console.warn -> error for non-streaming
routes, so a downgrade to log.warn wouldn't help; only the unexpected
shape (5xx, network) is worth surfacing.

* fix(#742): stop logging input verbatim from yes.op retry-failure paths

GitHub OAuth user tokens (ghu_...) were leaking to Better Stack on every
yes.op retry-failure for any utils/github/get* helper that takes a token
field — 38 leaks/7d in the most recent audit window. The leak path is
console.log inside the yes package (its own log shim, not utils/log.ts).

Drop input from the four log sites + the cache-key-derivation throw site.
key (SHA-1 of input) is sufficient for retry correlation; error already
carries request URL + status. Defense-in-depth comment so future
contributors don't re-add the field.

Operational follow-up (separate task): inventory ghu_... strings in
Better Stack ingested in the last 90d, revoke matching Clerk grants,
scrub cold-tier S3, rotate the BS source token.

* fix(#759): handle GraphqlResponseError "Could not resolve to a node" as 404

When the stored planCommentNodeId references a comment that's been
deleted on GitHub, octokit.graphql throws GraphqlResponseError before
the existing `node === null` 404 branch is reached. Add a narrow
isGraphqlNodeNotFound predicate in utils/errors.ts and a new catch
branch in the plan-comment route. The action treats 404 as "no prior
plan comment" and creates a fresh one, so behavior matches existing
contract.

* fix(#747): convert webhook GraphQL rate-limit 5xx into a Result<T> sentinel + 200 ack

When GitHub's GraphQL responds with "API rate limit exceeded for
installation ID N", _getReviewCommentsWithReplies threw, propagated
through the bare yes.op wrapper (no rate-limit bail), out of the bare
await in handleWebhook, and crashed /api/webhook/github with 500 — 77
webhook 500s/24h on the most recent audit window. GitHub redelivery
plus R2 dedup also silently masked the legitimate handler from
re-running once the rate-limit window cleared.

Mirror the #658 / _getRepository pattern: detect GraphqlResponseError
matching /rate limit (already )?exceeded/i, log.warn with the
x-ratelimit-reset value (and [Installation N] prefix when available),
return failure(...) with status 429. Webhook handler short-circuits
the case with 200 + log.info so GitHub stops the redelivery storm
against an exhausted budget, and the trigger page surfaces a clean
ThrowClientError. Document the new pattern as a Tier 2 false-positive
in wiki/log-audit.md so the next audit cron doesn't re-flag it.

Note that returning [] silently (the issue's first suggestion) would
have dropped @pullfrog mentions inline in review comments and
dispatched an agent run that re-rate-limits — skip-the-whole-case is
the correct semantics. Co-vulnerable getPullRequest / getWorkflow
have zero occurrences in this window; per #737 policy, defer until
they show up.

NOTE: this commit and the bracket of touched files revert as a unit —
the Result<T> shape change in getReviewCommentsWithReplies is
breaking; partial revert breaks the type chain.

* fix(#766): fold stderr+stdout into shell.ts errors + carve out merge-base --is-ancestor

action/utils/shell.ts dropped stdout when constructing failure messages
($\{stderr || "Unknown error"\}), so git subcommands that write
context-bearing diagnostics to stdout (merge conflicts, cherry-pick
rejections, diff --exit-code, ls-files --error-unmatch) surfaced as
"Command failed with exit code 1: Unknown error" through
mcp__pullfrog__git. The agent burned an extra MCP round-trip calling
git status to recover.

Fold stderr + stdout into the thrown error message (stderr first,
stdout fallback) so the agent always sees the real diagnostic. Plus
a narrow carve-out for `git merge-base --is-ancestor` in
action/mcp/git.ts: that subcommand uses exit code as data (0=ancestor,
1=not-an-ancestor, >1=error), so return { success: true, isAncestor }
instead of throwing on exit 1.

No caller in action/ string-matches on the old error format
(verified). diff --exit-code and ls-files --error-unmatch are not
carved out — both are zero-occurrence in the May audit window, and
the stderr+stdout fold renders their output usefully anyway.

* fix(#739): point customers at the actual fix when permissions: id-token: write is missing

When a customer workflow runs in GitHub Actions but lacks
permissions: id-token: write, ACTIONS_ID_TOKEN_REQUEST_URL/_TOKEN
aren't injected, isOIDCAvailable() is false, and acquireNewToken
falls through to the local-dev-only acquireTokenViaGitHubApp path,
which throws "GITHUB_APP_ID and GITHUB_PRIVATE_KEY must be set" —
pointing at a self-hosted-app fix that doesn't apply. One affected
customer burned 13 dispatches in 24h on this misleading error.

Detect (GITHUB_ACTIONS=true) AND (no OIDC env vars) inside
acquireNewToken before falling through to the local-dev branch, and
throw an actionable message naming the missing permissions block,
the exact YAML, and the docs anchor. The error surfaces via
##[error]action failed: ... in the workflow log (the only customer
surface available before main()'s inner try opens). Local-dev path
keeps the existing GITHUB_APP_ID message.

* fix(#760): suspend activity watchdog across in-flight tool calls

mcp__pullfrog__checkout_pr was hard-failing 6/24h on SenecaLabs/senecaWeb
because git fetch+deepen on a large monorepo can take 4-5 min, the
agent's stdout pipe goes silent the entire time (FastMCP is in-process
HTTP, but Claude/opencode CLIs await the synchronous tools/call
response), and both the spawn-level activity timer (300s in
subprocess.ts) and the process-level activity monitor (300s in
activity.ts) fire and kill the run.

Re-introduce the bracket pattern that PR #634 removed: bracket
suspendActivity()/resumeActivity() around tool_use -> tool_result in
both agent harnesses, plumb isPausedExternally into spawn() so both
timers suspend in lockstep. Bounded by MAX_TOOL_CALL_SUSPENSION_MS
(15 min auto-resume) plus the outer 1h agent timeout — neither
zombie-run avenue from #12 is reopened (subprocess.close still
resolves on death; outer timeout is suspend-agnostic; suspends gated
on explicit paired CLI events, not internal noise).

opencode tool_use handler: gate suspendActivity() on non-terminal
status (running/pending) so the bus_event re-dispatch path at line
915 — which only fires for completed/error subagent parts and never
emits a paired tool_result — doesn't latch the watchdog into
suspension until the 15min ceiling.

Add a heuristic:activity-watchdog-ceiling classifier to
scripts/analyze-logs.ts so a tool that genuinely hangs past
MAX_TOOL_CALL_SUSPENSION_MS surfaces in run-audit instead of being
bucketed into failure:unknown.

NOTE: this commit and the bracket of touched files revert as a unit
— activity.ts, subprocess.ts, and the two harnesses must move
together or the bracketing breaks.

* refactor(#747): swap Result<T> for InstallationRateLimitError typed throw

The Result<T> shape from 3ebf6c4c was cargo-culted from the #658
_getRepository pattern, but _getReviewCommentsWithReplies has only one
expected-error case (installation rate-limit) and two callers — Result
imposes branching on the trigger-page caller that never cared about
the rate-limit case specifically. A typed error class is lighter (~10
LoC vs ~33) and matches the actual need:

- new InstallationRateLimitError(resetAt) thrown from
  _getReviewCommentsWithReplies; rate-limit log.warn unchanged.
- handleWebhook catches it and breaks with log.info (unchanged
  semantics: 200 ack, no redelivery storm).
- trigger page reverts to direct array access; any failure propagates
  to the page error boundary (the pre-#747-commit shape).
- log-audit.md wording updated to match.
2026-05-19 18:40:53 +00:00

534 lines
16 KiB
TypeScript

import { createSign } from "node:crypto";
import { rename, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import * as core from "@actions/core";
import { throttling } from "@octokit/plugin-throttling";
import { Octokit } from "@octokit/rest";
import { apiFetch } from "./apiFetch.ts";
import { retry } from "./retry.ts";
function isObject(value: unknown) {
return typeof value === "object" && value !== null;
}
// we don't get access to the actual class from @octokit/rest
// it's reachable from @octokit/request-error but we'd have to add a dependency on it
// and it would pose a risk of accidentally pulling a different version of that class (node_modules dep graphs ❤️)
// so it's safer to ducktype this
interface OctokitResponseShim {
headers: Record<string, string | number | undefined>;
}
export interface InstallationToken {
token: string;
expires_at: string;
installation_id: number;
repository: string;
ref: string;
runner_environment: string;
owner?: string;
}
interface GitHubAppConfig {
appId: string;
privateKey: string;
repoOwner: string;
repoName: string;
}
interface Installation {
id: number;
account: {
login: string;
type: string;
};
}
interface Repository {
owner: {
login: string;
};
name: string;
}
interface InstallationTokenResponse {
token: string;
expires_at: string;
}
interface RepositoriesResponse {
repositories: Repository[];
}
function isOIDCAvailable(): boolean {
// OIDC requires both env vars to be set (only in real GitHub Actions with id-token permission)
return Boolean(
process.env.ACTIONS_ID_TOKEN_REQUEST_URL && process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN
);
}
type ReadWrite = "read" | "write";
type WriteOnly = "write";
/**
* GitHub App installation access token permissions.
* passed to `POST /app/installations/{id}/access_tokens` to scope the token.
* fields and allowed values come from the `app-permissions` OpenAPI schema.
* @see https://docs.github.com/en/rest/apps/installations#create-an-installation-access-token-for-an-app
* @see https://github.com/github/rest-api-description — components.schemas.app-permissions
*/
type GitHubAppPermissions = {
actions?: ReadWrite;
artifact_metadata?: ReadWrite;
attestations?: ReadWrite;
checks?: ReadWrite;
contents?: ReadWrite;
deployments?: ReadWrite;
discussions?: ReadWrite;
issues?: ReadWrite;
packages?: ReadWrite;
pages?: ReadWrite;
pull_requests?: ReadWrite;
security_events?: ReadWrite;
statuses?: ReadWrite;
workflows?: WriteOnly;
};
type AcquireTokenOptions = {
repos?: string[];
permissions?: GitHubAppPermissions;
};
/**
* Thrown when our token-exchange endpoint returns a non-2xx response.
* The retry policy in `acquireNewToken` looks for this concrete type to
* skip retries — 4xx is terminal user state (not-installed, not-authorized)
* and 5xx is rare enough that re-running the workflow is the right escape
* hatch. Genuine network failures throw plain `Error` and stay retryable.
*/
class TokenExchangeError extends Error {
readonly status: number;
constructor(status: number, message: string) {
super(message);
this.name = "TokenExchangeError";
this.status = status;
}
}
async function acquireTokenViaOIDC(opts?: AcquireTokenOptions): Promise<string> {
const oidcToken = await core.getIDToken("pullfrog-api");
const repos = [...(opts?.repos ?? [])];
const targetRepo = process.env.GITHUB_REPOSITORY?.split("/")[1];
if (targetRepo) {
repos.push(targetRepo);
}
const reposParam = repos.length ? `?repos=${repos.join(",")}` : "";
const timeoutMs = 30000;
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
try {
const tokenResponse = await apiFetch({
path: `/api/github/installation-token${reposParam}`,
method: "POST",
headers: {
Authorization: `Bearer ${oidcToken}`,
"Content-Type": "application/json",
},
body: opts?.permissions ? JSON.stringify({ permissions: opts.permissions }) : undefined,
signal: controller.signal,
});
clearTimeout(timeoutId);
if (!tokenResponse.ok) {
// prefer the server-side `error` field — it's the single source of
// truth for the install URL (uses GITHUB_APP_INSTALL_URL, which
// varies per env / GITHUB_APP_SLUG). fall back to a generic message
// if the body isn't JSON or doesn't carry an `error` field.
let serverMessage: string | undefined;
try {
const body = (await tokenResponse.json()) as { error?: unknown };
if (typeof body.error === "string") serverMessage = body.error;
} catch {
// body wasn't JSON — fall through to the generic message
}
throw new TokenExchangeError(
tokenResponse.status,
serverMessage ??
`Token exchange failed: ${tokenResponse.status} ${tokenResponse.statusText}`
);
}
const tokenData = (await tokenResponse.json()) as InstallationToken;
return tokenData.token;
} catch (error) {
clearTimeout(timeoutId);
if (error instanceof Error && error.name === "AbortError") {
throw new Error(`Token exchange timed out after ${timeoutMs}ms`);
}
throw error;
}
}
const base64UrlEncode = (str: string): string => {
return Buffer.from(str)
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
};
const generateJWT = (appId: string, privateKey: string): string => {
const now = Math.floor(Date.now() / 1000);
const payload = {
iat: now - 60,
exp: now + 5 * 60,
iss: appId,
};
const header = {
alg: "RS256",
typ: "JWT",
};
const encodedHeader = base64UrlEncode(JSON.stringify(header));
const encodedPayload = base64UrlEncode(JSON.stringify(payload));
const signaturePart = `${encodedHeader}.${encodedPayload}`;
const signature = createSign("RSA-SHA256")
.update(signaturePart)
.sign(privateKey, "base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
return `${signaturePart}.${signature}`;
};
const githubRequest = async <T>(
path: string,
options: {
method?: string;
headers?: Record<string, string>;
body?: string;
} = {}
): Promise<T> => {
const { method = "GET", headers = {}, body } = options;
const url = `https://api.github.com${path}`;
const requestHeaders = {
Accept: "application/vnd.github.v3+json",
"User-Agent": "Pullfrog-Installation-Token-Generator/1.0",
...headers,
};
const response = await fetch(url, {
method,
headers: requestHeaders,
...(body && { body }),
});
if (!response.ok) {
const errorText = await response.text();
throw new Error(
`GitHub API request failed: ${response.status} ${response.statusText}\n${errorText}`
);
}
return response.json() as T;
};
const checkRepositoryAccess = async (
token: string,
repoOwner: string,
repoName: string
): Promise<boolean> => {
try {
const response = await githubRequest<RepositoriesResponse>("/installation/repositories", {
headers: { Authorization: `token ${token}` },
});
const ownerLower = repoOwner.toLowerCase();
const nameLower = repoName.toLowerCase();
return response.repositories.some(
(repo) =>
repo.owner.login.toLowerCase() === ownerLower && repo.name.toLowerCase() === nameLower
);
} catch {
return false;
}
};
const createInstallationToken = async (
jwt: string,
installationId: number,
permissions?: GitHubAppPermissions
): Promise<string> => {
const requestOpts: { method: string; headers: Record<string, string>; body?: string } = {
method: "POST",
headers: { Authorization: `Bearer ${jwt}` },
};
if (permissions) {
requestOpts.body = JSON.stringify({ permissions });
}
const response = await githubRequest<InstallationTokenResponse>(
`/app/installations/${installationId}/access_tokens`,
requestOpts
);
return response.token;
};
const findInstallationId = async (
jwt: string,
repoOwner: string,
repoName: string
): Promise<number> => {
const installations = await githubRequest<Installation[]>("/app/installations", {
headers: { Authorization: `Bearer ${jwt}` },
});
for (const installation of installations) {
try {
const tempToken = await createInstallationToken(jwt, installation.id);
const hasAccess = await checkRepositoryAccess(tempToken, repoOwner, repoName);
if (hasAccess) {
return installation.id;
}
} catch {}
}
throw new Error(
`No installation found with access to ${repoOwner}/${repoName}. ` +
"Ensure the GitHub App is installed on the target repository."
);
};
// for local development only
async function acquireTokenViaGitHubApp(opts?: AcquireTokenOptions): Promise<string> {
if (!process.env.GITHUB_APP_ID || !process.env.GITHUB_PRIVATE_KEY) {
throw new Error(
"cannot acquire token via GitHub App: GITHUB_APP_ID and GITHUB_PRIVATE_KEY must be set"
);
}
const repoContext = parseRepoContext();
const config: GitHubAppConfig = {
appId: process.env.GITHUB_APP_ID,
privateKey: process.env.GITHUB_PRIVATE_KEY.replace(/\\n/g, "\n"),
repoOwner: repoContext.owner,
repoName: repoContext.name,
};
const jwt = generateJWT(config.appId, config.privateKey);
const installationId = await findInstallationId(jwt, config.repoOwner, config.repoName);
return await createInstallationToken(jwt, installationId, opts?.permissions);
}
/**
* ensure a GitHub token is available in the environment.
*
* when OIDC is available (CI), always mints a fresh token scoped to
* GITHUB_REPOSITORY — overriding any inherited GITHUB_TOKEN that may
* be scoped to the wrong repo.
*
* otherwise falls back to GitHub App credentials for local development.
*
* only called from play.ts (test/dev path) — the live action calls
* main() directly and never calls this.
*/
export async function ensureGitHubToken(): Promise<void> {
// when OIDC is available, always mint a fresh token scoped to
// GITHUB_REPOSITORY. the inherited GITHUB_TOKEN may be scoped to a
// different repo (e.g., runner token for pullfrog/app when tests
// target pullfrog/test-repo).
if (isOIDCAvailable()) {
const token = await acquireNewToken();
process.env.GITHUB_TOKEN = token;
return;
}
if (!process.env.GITHUB_TOKEN && !process.env.GH_TOKEN) {
const token = await acquireNewToken();
process.env.GITHUB_TOKEN = token;
}
}
export async function acquireNewToken(opts?: AcquireTokenOptions): Promise<string> {
if (isOIDCAvailable()) {
return await retry(() => acquireTokenViaOIDC(opts), {
label: "token exchange",
shouldRetry: (error) => {
// 4xx is terminal user state (app not installed, permissions wrong) —
// retrying just triples our log noise and the user's CI bill (see
// #693). 5xx/429 are transient (vercel cold start, github outage,
// rate limit) and should ride the existing backoff.
if (error instanceof TokenExchangeError) return error.status >= 500 || error.status === 429;
return (
error instanceof Error &&
(error.message.includes("timed out") ||
error.message.includes("fetch failed") ||
error.message.includes("ECONNRESET") ||
error.message.includes("ETIMEDOUT"))
);
},
});
}
// running inside GitHub Actions but the OIDC env vars are absent — the
// workflow is missing `permissions: id-token: write`. surface an
// actionable, customer-facing message; the GitHub-App branch below is
// local-dev only. see #739.
if (process.env.GITHUB_ACTIONS === "true") {
throw new Error(
"missing `permissions: id-token: write` on the Pullfrog workflow job.\n" +
"\n" +
"Pullfrog mints short-lived GitHub App installation tokens via OIDC and\n" +
"requires `id-token: write` to be granted at the job level. add the\n" +
"following to your workflow yaml:\n" +
"\n" +
" jobs:\n" +
" pullfrog:\n" +
" permissions:\n" +
" id-token: write # mint Pullfrog installation tokens via OIDC\n" +
" contents: read # for actions/checkout\n" +
"\n" +
"see https://docs.pullfrog.com/headless-action#required-permissions for the full template."
);
}
// local development via GitHub App
return await acquireTokenViaGitHubApp(opts);
}
export interface RepoContext {
owner: string;
name: string;
}
/**
* Parse repository context from GITHUB_REPOSITORY environment variable.
*/
export function parseRepoContext(): RepoContext {
const githubRepo = process.env.GITHUB_REPOSITORY;
if (!githubRepo) {
throw new Error("GITHUB_REPOSITORY environment variable is required");
}
const [owner, name] = githubRepo.split("/");
if (!owner || !name) {
throw new Error(`Invalid GITHUB_REPOSITORY format: ${githubRepo}. Expected 'owner/repo'`);
}
return { owner, name };
}
export type OctokitWithPlugins = InstanceType<
ReturnType<typeof Octokit.plugin<typeof Octokit, [typeof throttling]>>
>;
export interface ResourceUsage {
requestCount: number;
rateLimitRemaining: number | null;
rateLimitResetMs: number | null;
}
function emptyResourceUsage(): ResourceUsage {
return {
requestCount: 0,
rateLimitRemaining: null,
rateLimitResetMs: null,
};
}
const usageByResource: Record<string, ResourceUsage> = {
core: emptyResourceUsage(),
graphql: emptyResourceUsage(),
};
export interface UsageSummary {
version: 1;
github: {
core: ResourceUsage;
graphql: ResourceUsage;
};
}
function getGitHubUsageSummary(): UsageSummary {
return {
version: 1,
github: {
core: usageByResource.core,
graphql: usageByResource.graphql,
},
};
}
export async function writeGitHubUsageSummaryToFile(path: string): Promise<void> {
const summary = getGitHubUsageSummary();
const tmpPath = join(dirname(path), `.usage-summary-${process.pid}.tmp`);
await writeFile(tmpPath, JSON.stringify(summary));
await rename(tmpPath, path);
}
export function createOctokit(token: string): OctokitWithPlugins {
// `OctokitWithPlugins` initialization based on https://github.com/actions/toolkit/blob/2506e78e82fbd2f9e94d63e75f5309118c8de1b1/packages/github/src/github.ts#L15-L22
// we can't use it directly because it's stuck on `@octokit/core@v5` and we use the hottest `@octokit/core@v7`
const OctokitWithPlugins = Octokit.plugin(throttling);
const octokit = new OctokitWithPlugins({
auth: token,
throttle: {
onRateLimit: (_retryAfter, _options, _octokit, retryCount) => {
return retryCount <= 2;
},
onSecondaryRateLimit: (_retryAfter, _options, _octokit, retryCount) => {
return retryCount <= 2;
},
},
});
const onResponse = (response: OctokitResponseShim) => {
const resource = response.headers["x-ratelimit-resource"];
if (!resource) {
return response;
}
usageByResource[resource] ??= emptyResourceUsage();
const usage = usageByResource[resource];
usage.requestCount++;
const remaining = response.headers["x-ratelimit-remaining"];
const reset = response.headers["x-ratelimit-reset"];
if (remaining !== undefined) {
usage.rateLimitRemaining = Number(remaining);
}
if (reset !== undefined) {
usage.rateLimitResetMs = Number(reset) * 1000;
}
return response;
};
octokit.hook.wrap("request", async (request, options) => {
try {
const response = await request(options);
onResponse(response);
return response;
} catch (error) {
if (
isObject(error) &&
"response" in error &&
isObject(error.response) &&
"headers" in error.response &&
isObject(error.response.headers)
) {
onResponse(error.response as OctokitResponseShim);
}
throw error;
}
});
return octokit;
}