feat: Amazon Bedrock support via routing slug (#720)
* add Amazon Bedrock as a routing slug introduces a single `bedrock/byok` catalog entry that the harness translates to the appropriate Bedrock model ID at run time via `BEDROCK_MODEL_ID`. routes Anthropic IDs through claude-code (with `CLAUDE_CODE_USE_BEDROCK=1`) and everything else through opencode's `amazon-bedrock` provider — keeps the catalog flat for an audience that needs version pinning rather than aliasing. accepts either `AWS_BEARER_TOKEN_BEDROCK` or `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` for auth; both validated alongside `AWS_REGION` and `BEDROCK_MODEL_ID` in `validateAgentApiKey`. catalog drift tests, the bumps cron, and per-alias smoke scripts all skip routing slugs since there's no fixed `resolve` to validate. docs/bedrock.mdx walks through setup; wiki/model-resolution.md has a section explaining why bedrock breaks the usual alias pattern. closes pullfrog/pullfrog#40 * ci: add bedrock env vars to test workflows mirrors the new bedrock provider's required env vars (AWS_BEARER_TOKEN_BEDROCK inherited from org secret + AWS_REGION + BEDROCK_MODEL_ID hardcoded) into both .github/workflows/test.yml files so the ci.test "env vars cover all provider API keys" assertion passes. * docs(bedrock): clearer setup flow + screenshot of model selector restructures the setup section into three concrete steps in execution order: select Bedrock from the dropdown, store the bearer token as a secret (Pullfrog or GitHub — links to keys.mdx for the trade-off), then add region + model id directly in pullfrog.yml since neither is sensitive. enable-model-access in the Bedrock console moved to step 4 (only required once per model and only when AWS rejects the call, not blocking on first run). adds a screenshot of the console model selector with Amazon Bedrock selected so readers can recognize the UI state they're aiming for. * fix(bedrock): tolerate raw Bedrock model IDs in validateAgentApiKey main.ts passes the resolved model into validateAgentApiKey (`payload.proxyModel ?? resolvedModel ?? payload.model`). For Bedrock, `resolveModel` translates `bedrock/byok` into the raw AWS model ID (e.g. `us.anthropic.claude-opus-4-6-v1`), which has no `/` and so trips parseModel inside getModelEnvVars. Detect the no-slash case and re-run the bedrock setup check (auth + region; BEDROCK_MODEL_ID is already enforced upstream by resolveModel). Caught by PR #720 e2e dispatch on pullfrog/preview-720-bedrock — "invalid model slug 'us.anthropic.claude-opus-4-6-v1' — expected 'provider/model'". Two regression tests cover the raw-ID path. * fix(bedrock): always prepend amazon-bedrock/ prefix when bedrock-routed opencode.ts was gating the prefix-injection on `!isBedrockAnthropicId(rawModel)`, on the theory that Anthropic Bedrock IDs always go through claude-code. But `PULLFROG_AGENT=opencode` is a documented escape hatch — when it forces opencode for an Anthropic Bedrock model, the prefix still has to be added or opencode fails with 'Model not found: <modelId>/.'. The Anthropic-vs-other discriminant only belongs in resolveAgent. Once an agent is selected, it should consistently honor the bedrock route. Caught by the PULLFROG_AGENT=opencode + Opus 4.6 e2e on pullfrog/preview-720-bedrock — run 25823437606. * ui+docs(bedrock): bespoke setup callout + clearer docs UI: - BedrockSetupCallout in components/AgentSettings.tsx covers both the Model costs section and the onboarding card. Detects bedrock via resolveDisplayAlias().routing === "bedrock", shows a dedicated message ("store AWS_BEARER_TOKEN_BEDROCK as a secret, then put AWS_REGION + BEDROCK_MODEL_ID directly in pullfrog.yml") + link to the setup guide. Replaces the generic "X, Y, or Z is required" prompt that misrepresented the three values as three separate secrets to add (and used the wrong "or" connector for what's actually an AND). - OnboardingCard re-uses the same callout with the gradient-card variant. Docs: - Drop the obsolete "Enable model access" step. AWS retired the manual enrollment page; foundation models auto-enable on first invocation. Anthropic models still need a one-time use-case form for first-time users — surfaced under the AccessDenied troubleshooting entry. - Drop the "Testing a different model in one run" PULLFROG_MODEL note. It introduced the secrets-vs-vars distinction we want to keep out of the bedrock setup story. - Step 3 already recommends hardcoding region + model id in pullfrog.yml. Workflow template: - The default pullfrog.yml customers receive (utils/github/pullfrog.yml.ts) now references AWS_BEARER_TOKEN_BEDROCK from secrets but inlines AWS_REGION and BEDROCK_MODEL_ID as plain values. Matches the docs. * fix(bedrock): three review-caught edges in routing + UI copy Addresses three real issues from PR #720 review: 1. agent.ts: PULLFROG_MODEL=bedrock/byok no longer leaks the literal sentinel "bedrock" downstream. resolveCliModel returns the alias's resolve field verbatim, which for routing entries IS the sentinel. Refactored both the env-override and slug-lookup paths through a shared resolveSlug() that recognizes routing aliases and defers to their backing env var (BEDROCK_MODEL_ID). 2. models.ts: isBedrockAnthropicId() now anchors on a discrete dot/slash/colon-segment match (case-insensitive) instead of a substring contains. The substring check was fragile in both directions for inference-profile ARNs (BEDROCK_MODEL_ID accepts ARNs per AWS docs) — a non-Anthropic profile whose user-chosen name contained "anthropic" would mis-route to claude-code, and an Anthropic profile whose name omitted it would miss CLAUDE_CODE_USE_BEDROCK=1. 3. AgentSettings.tsx: BedrockSetupCallout's configured-state copy showed "AWS_BEARER_TOKEN_BEDROCK configured" even when the user satisfied the gate via AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY, gaslighting access-key users about a secret they never set. Detect which auth method is actually present and name the right secret(s) in the success message. Regression tests in models.test.ts (5 new isBedrockAnthropicId cases including positive and negative ARN forms) and agent.test.ts (2 new PULLFROG_MODEL=bedrock/byok cases). 171/171 action tests pass. * yml template: add commented AWS access-key alternative for Bedrock auth Mirrors the IAM access-key path verified end-to-end on PR #720 e2e run 25830764987. Bearer token stays as the primary nudge; the access-key pair is the fallback for users who can't mint Bedrock API keys. * yml template: drop redundant 'or, alternatively' annotation * ui+docs(bedrock): rewrite callout copy + refresh screenshot Reframes the BedrockSetupCallout away from generic BYOK language to a Bedrock-specific message: leads with "Amazon Bedrock is configured entirely via environment variables", lists all four (auth, region, model id), and ends with the requested CTA sentence ("click below to learn more about Bedrock support in Pullfrog"). Promotes the "Bedrock setup guide" docs link from an inline anchor to a prominent button (always visible, regardless of auth state). The "Add AWS_BEARER_TOKEN_BEDROCK" affordance is now a secondary chip shown only when no auth secret is configured. Refreshes docs/images/model-selector-bedrock.png to capture the new callout — the prior screenshot still showed the old generic "BYOK / X, Y, or Z required" wording.
This commit is contained in:
committed by
pullfrog[bot]
parent
1a9d3c1f82
commit
8f9208bd3f
@@ -55,6 +55,9 @@ jobs:
|
||||
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|
||||
MOONSHOT_API_KEY: ${{ secrets.MOONSHOT_API_KEY }}
|
||||
OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
|
||||
AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
|
||||
AWS_REGION: us-east-1
|
||||
BEDROCK_MODEL_ID: us.anthropic.claude-opus-4-6-v1
|
||||
PULLFROG_MODEL: ${{ vars.PULLFROG_MODEL }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
+31
-1
@@ -16,6 +16,7 @@ import { mkdirSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { pullfrogMcpName } from "../external.ts";
|
||||
import { BEDROCK_MODEL_ID_ENV, isBedrockAnthropicId } from "../models.ts";
|
||||
|
||||
import { getIdleMs, markActivity } from "../utils/activity.ts";
|
||||
import { formatJsonValue, log } from "../utils/cli.ts";
|
||||
@@ -844,7 +845,22 @@ export const claude = agent({
|
||||
const cliPath = await installClaudeCli();
|
||||
|
||||
const specifier = ctx.payload.proxyModel ?? ctx.resolvedModel;
|
||||
const model = specifier ? stripProviderPrefix(specifier) : undefined;
|
||||
// claude-code on Bedrock takes the bare AWS model ID — no provider prefix
|
||||
// to strip, since the ID is already in `provider.model` form (e.g.
|
||||
// `us.anthropic.claude-opus-4-7`). detect via the env-var sentinel: if
|
||||
// BEDROCK_MODEL_ID is set and matches the resolved specifier, this is a
|
||||
// bedrock route. see `wiki/model-resolution.md` for the routing pattern.
|
||||
const bedrockModelId = process.env[BEDROCK_MODEL_ID_ENV]?.trim();
|
||||
const isBedrockRoute =
|
||||
specifier !== undefined &&
|
||||
bedrockModelId !== undefined &&
|
||||
bedrockModelId === specifier &&
|
||||
isBedrockAnthropicId(specifier);
|
||||
const model = !specifier
|
||||
? undefined
|
||||
: isBedrockRoute
|
||||
? specifier
|
||||
: stripProviderPrefix(specifier);
|
||||
|
||||
const homeEnv = {
|
||||
HOME: ctx.tmpdir,
|
||||
@@ -891,10 +907,24 @@ export const claude = agent({
|
||||
|
||||
// agent process gets full env — needs LLM API keys, PATH, locale, etc.
|
||||
// security is enforced via managed-settings.json, --disallowedTools (Bash), and MCP tool filtering.
|
||||
//
|
||||
// bedrock route: claude-code reads `CLAUDE_CODE_USE_BEDROCK=1` to switch
|
||||
// its provider implementation from the direct Anthropic API to Bedrock.
|
||||
// AWS_BEARER_TOKEN_BEDROCK / AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY +
|
||||
// AWS_REGION are already in process.env from the workflow's `env:` block.
|
||||
// see https://docs.claude.com/en/docs/claude-code/amazon-bedrock.
|
||||
//
|
||||
// we only force CLAUDE_CODE_USE_BEDROCK=1 when this is a Pullfrog-routed
|
||||
// bedrock run; if the user has set the env var manually for some other
|
||||
// reason (e.g. always-Bedrock org policy), `...process.env` already
|
||||
// carries it through and we don't disturb it.
|
||||
const env: Record<string, string | undefined> = {
|
||||
...process.env,
|
||||
...homeEnv,
|
||||
};
|
||||
if (isBedrockRoute) {
|
||||
env.CLAUDE_CODE_USE_BEDROCK = "1";
|
||||
}
|
||||
|
||||
const repoDir = process.cwd();
|
||||
|
||||
|
||||
+19
-2
@@ -16,7 +16,7 @@ import { mkdirSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { pullfrogMcpName } from "../external.ts";
|
||||
import { modelAliases } from "../models.ts";
|
||||
import { BEDROCK_MODEL_ID_ENV, modelAliases } from "../models.ts";
|
||||
import { getIdleMs, markActivity } from "../utils/activity.ts";
|
||||
import { formatJsonValue, log } from "../utils/cli.ts";
|
||||
import { installFromNpmTarball } from "../utils/install.ts";
|
||||
@@ -1175,7 +1175,24 @@ export const opencode = agent({
|
||||
run: async (ctx) => {
|
||||
const cliPath = await installOpencodeCli();
|
||||
|
||||
const model = ctx.payload.proxyModel ?? ctx.resolvedModel ?? autoSelectModel(cliPath);
|
||||
const rawModel = ctx.payload.proxyModel ?? ctx.resolvedModel ?? autoSelectModel(cliPath);
|
||||
|
||||
// bedrock route: opencode's `amazon-bedrock` provider expects the model
|
||||
// string in `amazon-bedrock/<bedrock-id>` form. the bare AWS model ID
|
||||
// (what the user puts in `BEDROCK_MODEL_ID`) needs the prefix added.
|
||||
// detect via env-var sentinel — same pattern as claude.ts.
|
||||
//
|
||||
// we deliberately do NOT gate on `!isBedrockAnthropicId(rawModel)` here:
|
||||
// Anthropic-on-Bedrock normally routes to claude-code (per `resolveAgent`),
|
||||
// but `PULLFROG_AGENT=opencode` is the documented escape hatch for forcing
|
||||
// opencode regardless. when that override fires, opencode still needs the
|
||||
// `amazon-bedrock/` prefix or the provider lookup fails with
|
||||
// "Model not found: <modelId>/.". the Anthropic-vs-other discriminant
|
||||
// only belongs in `resolveAgent`.
|
||||
const bedrockModelId = process.env[BEDROCK_MODEL_ID_ENV]?.trim();
|
||||
const isBedrockRoute =
|
||||
rawModel !== undefined && bedrockModelId !== undefined && bedrockModelId === rawModel;
|
||||
const model = isBedrockRoute ? `amazon-bedrock/${rawModel}` : rawModel;
|
||||
|
||||
const homeEnv = {
|
||||
HOME: ctx.tmpdir,
|
||||
|
||||
+9
-2
@@ -22,9 +22,16 @@ type CliProvider = {
|
||||
|
||||
function buildProviders(): CliProvider[] {
|
||||
return Object.entries(providers)
|
||||
.filter(([key]) => key !== "opencode" && key !== "openrouter")
|
||||
.filter(([key]) => key !== "opencode" && key !== "openrouter" && key !== "bedrock")
|
||||
.map(([key, config]: [string, ProviderConfig]) => {
|
||||
const aliases = modelAliases.filter((a) => a.provider === key && !a.fallback && !a.hidden);
|
||||
// bedrock requires multi-secret setup (auth + region + model id) that
|
||||
// doesn't fit the single-paste flow below — direct users to
|
||||
// https://docs.pullfrog.com/bedrock instead. revisit once the init flow
|
||||
// supports multi-value setup. `hidden` excludes internal-only subagent
|
||||
// targets (e.g. openai/gpt-5.4) per #710.
|
||||
const aliases = modelAliases.filter(
|
||||
(a) => a.provider === key && !a.fallback && !a.routing && !a.hidden
|
||||
);
|
||||
const recommended = aliases.find((a) => a.preferred);
|
||||
const sorted = [...aliases].sort((a, b) => {
|
||||
if (a.preferred && !b.preferred) return -1;
|
||||
|
||||
+59
-1
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
getModelEnvVars,
|
||||
getModelProvider,
|
||||
isBedrockAnthropicId,
|
||||
modelAliases,
|
||||
parseModel,
|
||||
providers,
|
||||
@@ -175,7 +176,12 @@ describe("modelAliases registry", () => {
|
||||
|
||||
it("has exactly one preferred model per provider", () => {
|
||||
for (const providerKey of Object.keys(providers)) {
|
||||
const preferred = modelAliases.filter((a) => a.provider === providerKey && a.preferred);
|
||||
// routing-only providers (bedrock) deliberately have no preferred
|
||||
// model — the user picks the actual model via a per-run env var, so
|
||||
// there's no "preferred default" to surface to auto-select.
|
||||
const aliases = modelAliases.filter((a) => a.provider === providerKey);
|
||||
if (aliases.every((a) => a.routing)) continue;
|
||||
const preferred = aliases.filter((a) => a.preferred);
|
||||
expect(preferred.length, `${providerKey} should have exactly 1 preferred model`).toBe(1);
|
||||
}
|
||||
});
|
||||
@@ -190,6 +196,10 @@ describe("modelAliases registry", () => {
|
||||
|
||||
it("all resolve values follow provider/model format", () => {
|
||||
for (const alias of modelAliases) {
|
||||
// routing slugs use a sentinel `resolve` (e.g. "bedrock") that's never
|
||||
// passed to a CLI directly — the harness reads a separate env var to
|
||||
// get the real model ID. format check doesn't apply.
|
||||
if (alias.routing) continue;
|
||||
expect(alias.resolve).toContain("/");
|
||||
}
|
||||
});
|
||||
@@ -200,6 +210,54 @@ describe("modelAliases registry", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("isBedrockAnthropicId", () => {
|
||||
it("matches geo-prefixed Anthropic foundation IDs", () => {
|
||||
expect(isBedrockAnthropicId("us.anthropic.claude-opus-4-7")).toBe(true);
|
||||
expect(isBedrockAnthropicId("eu.anthropic.claude-sonnet-4-6")).toBe(true);
|
||||
expect(isBedrockAnthropicId("global.anthropic.claude-haiku-4-5-20251001-v1:0")).toBe(true);
|
||||
});
|
||||
|
||||
it("matches in-region Anthropic foundation IDs", () => {
|
||||
expect(isBedrockAnthropicId("anthropic.claude-opus-4-7")).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects non-Anthropic foundation IDs", () => {
|
||||
expect(isBedrockAnthropicId("amazon.nova-pro-v1:0")).toBe(false);
|
||||
expect(isBedrockAnthropicId("us.meta.llama4-scout-17b-instruct-v1:0")).toBe(false);
|
||||
expect(isBedrockAnthropicId("deepseek.v3.2")).toBe(false);
|
||||
});
|
||||
|
||||
// regression: PR #720 review caught that a substring-only match was
|
||||
// fragile for inference-profile ARNs (which BEDROCK_MODEL_ID accepts per
|
||||
// the AWS docs). ARN names are user-chosen — both directions of the
|
||||
// heuristic could break depending on what name the operator picked.
|
||||
// We anchor on a discrete dot-segment match (case-insensitive) instead.
|
||||
it("ignores 'anthropic' substrings inside non-segment text", () => {
|
||||
// ARN whose user-chosen profile name happens to contain "anthropic" as
|
||||
// part of a longer word — would route to claude-code under naive
|
||||
// includes("anthropic") even though the backing model is unknown.
|
||||
expect(
|
||||
isBedrockAnthropicId(
|
||||
"arn:aws:bedrock:us-east-2:123456789012:application-inference-profile/my-anthropicish-profile"
|
||||
)
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("matches when 'anthropic' appears as its own dot-segment in ARN", () => {
|
||||
// ARN whose profile name embeds the foundation segment correctly —
|
||||
// operator chose to surface the backing model in the name.
|
||||
expect(
|
||||
isBedrockAnthropicId(
|
||||
"arn:aws:bedrock:us-east-2:123456789012:application-inference-profile/anthropic.claude-opus-4-7"
|
||||
)
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("is case-insensitive", () => {
|
||||
expect(isBedrockAnthropicId("US.ANTHROPIC.CLAUDE-OPUS-4-7")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("providers registry", () => {
|
||||
it("every provider has envVars", () => {
|
||||
for (const [key, config] of Object.entries(providers)) {
|
||||
|
||||
@@ -7,6 +7,22 @@
|
||||
|
||||
// ── types ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* routing discriminant for entries whose `resolve` is dynamic — looked up
|
||||
* from a separate env var at run time rather than fixed in the catalog.
|
||||
*
|
||||
* `"bedrock"` means the actual model ID comes from `BEDROCK_MODEL_ID`
|
||||
* (an AWS-canonical Bedrock model ID like `us.anthropic.claude-opus-4-7`
|
||||
* or `amazon.nova-pro-v1:0`). enterprise Bedrock customers self-select for
|
||||
* version control — silent alias bumps would break compliance review,
|
||||
* model-access enrollment, and provisioned-throughput contracts. so the
|
||||
* single `bedrock/byok` entry is a routing slug, not a model alias: the
|
||||
* harness reads `BEDROCK_MODEL_ID` and routes to claude-code (when the ID
|
||||
* contains "anthropic") or opencode (everything else, with an
|
||||
* `amazon-bedrock/` prefix).
|
||||
*/
|
||||
export type ModelRouting = "bedrock";
|
||||
|
||||
export interface ModelAlias {
|
||||
/** stable alias stored in DB, e.g. "anthropic/claude-opus" */
|
||||
slug: string;
|
||||
@@ -14,9 +30,9 @@ export interface ModelAlias {
|
||||
provider: string;
|
||||
/** human-readable name shown in dropdowns */
|
||||
displayName: string;
|
||||
/** concrete models.dev specifier, e.g. "anthropic/claude-opus-4-6" */
|
||||
/** concrete models.dev specifier, e.g. "anthropic/claude-opus-4-6". sentinel for routing entries — never passed to a CLI directly. */
|
||||
resolve: string;
|
||||
/** full models.dev specifier for the OpenRouter equivalent (undefined for free models) */
|
||||
/** full models.dev specifier for the OpenRouter equivalent (undefined for free models and routing entries) */
|
||||
openRouterResolve: string | undefined;
|
||||
/** top-tier pick for this provider — preferred during auto-select */
|
||||
preferred: boolean;
|
||||
@@ -24,6 +40,8 @@ export interface ModelAlias {
|
||||
isFree: boolean;
|
||||
/** slug of a replacement model — presence implies this model is deprecated */
|
||||
fallback: string | undefined;
|
||||
/** dynamic-resolution discriminant — see ModelRouting docs */
|
||||
routing: ModelRouting | undefined;
|
||||
/** alias key (within same provider) of the cheaper sibling reviewfrog should
|
||||
* use as its lens-fanout subagent. e.g. claude-opus → "claude-sonnet". */
|
||||
subagentModel: string | undefined;
|
||||
@@ -44,6 +62,8 @@ interface ModelDef {
|
||||
isFree?: boolean;
|
||||
/** slug of a replacement model — presence implies this model is deprecated */
|
||||
fallback?: string;
|
||||
/** dynamic-resolution discriminant — see ModelRouting docs */
|
||||
routing?: ModelRouting;
|
||||
/** alias key (within same provider) of the cheaper sibling reviewfrog should
|
||||
* use as its lens-fanout subagent (e.g. claude-opus → "claude-sonnet"). */
|
||||
subagentModel?: string;
|
||||
@@ -324,6 +344,20 @@ export const providers = {
|
||||
},
|
||||
},
|
||||
}),
|
||||
bedrock: provider({
|
||||
displayName: "Amazon Bedrock",
|
||||
envVars: ["AWS_BEARER_TOKEN_BEDROCK", "AWS_REGION", "BEDROCK_MODEL_ID"],
|
||||
models: {
|
||||
// single routing entry — the actual Bedrock model ID is read from
|
||||
// BEDROCK_MODEL_ID at run time. see ModelRouting docs for why we
|
||||
// don't catalog individual Bedrock models.
|
||||
byok: {
|
||||
displayName: "Amazon Bedrock",
|
||||
resolve: "bedrock",
|
||||
routing: "bedrock",
|
||||
},
|
||||
},
|
||||
}),
|
||||
openrouter: provider({
|
||||
displayName: "OpenRouter",
|
||||
envVars: ["OPENROUTER_API_KEY"],
|
||||
@@ -479,6 +513,7 @@ export const modelAliases: ModelAlias[] = Object.entries(providers).flatMap(
|
||||
preferred: def.preferred ?? false,
|
||||
isFree: def.isFree ?? false,
|
||||
fallback: def.fallback,
|
||||
routing: def.routing,
|
||||
// subagentModel is stored as an alias key local to the provider; expand
|
||||
// here to a fully-qualified slug so callers can look up the target alias
|
||||
// directly without re-deriving the provider.
|
||||
@@ -537,3 +572,40 @@ export function resolveCliModel(slug: string): string | undefined {
|
||||
export function resolveOpenRouterModel(slug: string): string | undefined {
|
||||
return resolveDisplayAlias(slug)?.openRouterResolve;
|
||||
}
|
||||
|
||||
// ── bedrock routing ────────────────────────────────────────────────────────────
|
||||
|
||||
/** env var that supplies the Bedrock model ID for the `bedrock/byok` slug. */
|
||||
export const BEDROCK_MODEL_ID_ENV = "BEDROCK_MODEL_ID";
|
||||
|
||||
/**
|
||||
* the Bedrock model ID passed to claude-code or opencode is whatever the
|
||||
* user set in `BEDROCK_MODEL_ID` — Pullfrog never resolves or upgrades it.
|
||||
* we route by checking whether the ID names an Anthropic model: claude-code
|
||||
* handles Anthropic-on-Bedrock natively (with `CLAUDE_CODE_USE_BEDROCK=1`),
|
||||
* everything else goes through opencode's `amazon-bedrock` provider.
|
||||
*
|
||||
* AWS Bedrock IDs come in two shapes:
|
||||
* - dotted foundation IDs: `us.anthropic.claude-opus-4-7`,
|
||||
* `anthropic.claude-haiku-4-5-20251001-v1:0`, `amazon.nova-pro-v1:0`,
|
||||
* `meta.llama4-scout-17b-instruct-v1:0`. AWS-published, lowercase, the
|
||||
* foundation provider always appears as a discrete dot-segment.
|
||||
* - inference-profile ARNs: `arn:aws:bedrock:us-east-2:<acct>:application-inference-profile/<user-name>`.
|
||||
* `<user-name>` is operator-chosen, so a naive substring check is fragile
|
||||
* in both directions (Anthropic profile named without "anthropic" → routes
|
||||
* to opencode and misses CLAUDE_CODE_USE_BEDROCK; non-Anthropic profile
|
||||
* whose name happens to contain "anthropic" → routes to claude-code).
|
||||
*
|
||||
* we anchor on a discrete dot-segment match (case-insensitive). this catches
|
||||
* every published foundation ID and is conservative for ARN-form IDs: ARN
|
||||
* names that don't include "anthropic" as their own dot-segment route to
|
||||
* opencode by default. operators using ARN-form IDs whose backing model is
|
||||
* Anthropic should set `PULLFROG_AGENT=claude` to force the right route, or
|
||||
* include the foundation segment in the profile name.
|
||||
*/
|
||||
export function isBedrockAnthropicId(bedrockModelId: string): boolean {
|
||||
// split on `.`, `/`, and `:` so the check works for both dotted foundation
|
||||
// IDs (anthropic.* / us.anthropic.*) and ARN-form IDs (where the relevant
|
||||
// foundation segment sits between `/` and `.` inside the resource name).
|
||||
return bedrockModelId.toLowerCase().split(/[./:]/).includes("anthropic");
|
||||
}
|
||||
|
||||
@@ -50,6 +50,9 @@ const FLAGSHIPS = [
|
||||
function isPrunablePassthrough(alias: (typeof modelAliases)[number]): boolean {
|
||||
if (ROUTING_CANARIES.has(alias.slug)) return false;
|
||||
if (alias.provider === "openrouter") return true;
|
||||
// routing slugs (bedrock/byok) need a per-run env var to pick the actual
|
||||
// model — there's no generic smoke test, so prune from both matrices.
|
||||
if (alias.routing) return true;
|
||||
// opencode FREE models (big-pickle, mimo-v2-pro-free, minimax-m2.5-free)
|
||||
// are unique to opencode and used in prod — keep them. only prune the keyed
|
||||
// mirrors.
|
||||
|
||||
@@ -44,6 +44,11 @@ type Plan =
|
||||
async function plan(slug: string): Promise<Plan> {
|
||||
const alias = modelAliases.find((a) => a.slug === slug);
|
||||
if (!alias) throw new Error(`model-smoke: unknown alias "${slug}"`);
|
||||
if (alias.routing) {
|
||||
throw new Error(
|
||||
`model-smoke: ${slug} is a routing slug (no fixed model). pass an explicit Bedrock model ID via PULLFROG_MODEL or the workflow env block.`
|
||||
);
|
||||
}
|
||||
|
||||
// walk the fallback chain so deprecated aliases (those with `fallback` set,
|
||||
// e.g. opencode/mimo-v2-pro-free → opencode/big-pickle) hit their replacement
|
||||
|
||||
@@ -42,6 +42,11 @@ describe("models.dev validity", async () => {
|
||||
const data = await api;
|
||||
|
||||
for (const alias of modelAliases) {
|
||||
// routing slugs (e.g. bedrock/byok) have no fixed `resolve` — the actual
|
||||
// model ID is read from a separate env var at run time. skip drift checks
|
||||
// since there's no models.dev entry to validate against.
|
||||
if (alias.routing) continue;
|
||||
|
||||
const parsed = parseResolve(alias.resolve);
|
||||
|
||||
it(`${alias.resolve} exists on models.dev`, () => {
|
||||
|
||||
@@ -14,6 +14,10 @@ const BYOK_ONLY_MODELS = new Set(["openai/o3"]);
|
||||
describe("openRouterResolve completeness", () => {
|
||||
for (const alias of modelAliases) {
|
||||
if (alias.isFree) continue;
|
||||
// routing slugs (e.g. bedrock/byok) are inherently BYOK — there's no
|
||||
// single model to map to OpenRouter because the actual model ID is read
|
||||
// from a per-run env var.
|
||||
if (alias.routing) continue;
|
||||
if (BYOK_ONLY_MODELS.has(alias.slug)) continue;
|
||||
it(`${alias.slug} has openRouterResolve`, () => {
|
||||
expect(
|
||||
@@ -29,6 +33,13 @@ describe("openRouterResolve completeness", () => {
|
||||
expect(alias.openRouterResolve).toBeUndefined();
|
||||
});
|
||||
}
|
||||
|
||||
for (const alias of modelAliases) {
|
||||
if (!alias.routing) continue;
|
||||
it(`${alias.slug} (routing slug) has no openRouterResolve`, () => {
|
||||
expect(alias.openRouterResolve).toBeUndefined();
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe("fallback chain resolution", () => {
|
||||
|
||||
@@ -28,6 +28,14 @@ const results: { model: string; status: "pass" | "fail" | "skip"; detail?: strin
|
||||
const seen = new Set<string>();
|
||||
|
||||
for (const alias of modelAliases) {
|
||||
// routing slugs (bedrock/byok) have no fixed `resolve` to test against —
|
||||
// the model ID is supplied at run time via a per-run env var. skipping
|
||||
// here matches the bumps cron + catalog drift test.
|
||||
if (alias.routing) {
|
||||
results.push({ model: alias.slug, status: "skip", detail: "routing slug (no fixed resolve)" });
|
||||
continue;
|
||||
}
|
||||
|
||||
if (seen.has(alias.resolve)) {
|
||||
results.push({ model: alias.resolve, status: "skip", detail: "duplicate resolve" });
|
||||
continue;
|
||||
|
||||
+127
-5
@@ -1,9 +1,131 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { resolveAgent } from "./agent.ts";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { resolveAgent, resolveModel } from "./agent.ts";
|
||||
|
||||
const savedEnv = { ...process.env };
|
||||
|
||||
const STRIPPED = [
|
||||
/_API_KEY$/,
|
||||
/^CLAUDE_CODE_OAUTH_TOKEN$/,
|
||||
/^AWS_BEARER_TOKEN_BEDROCK$/,
|
||||
/^AWS_ACCESS_KEY_ID$/,
|
||||
/^AWS_SECRET_ACCESS_KEY$/,
|
||||
/^AWS_SESSION_TOKEN$/,
|
||||
/^AWS_REGION$/,
|
||||
/^BEDROCK_MODEL_ID$/,
|
||||
/^PULLFROG_MODEL$/,
|
||||
/^PULLFROG_AGENT$/,
|
||||
];
|
||||
|
||||
beforeEach(() => {
|
||||
for (const key of Object.keys(process.env)) {
|
||||
if (STRIPPED.some((re) => re.test(key))) delete process.env[key];
|
||||
}
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env = { ...savedEnv };
|
||||
});
|
||||
|
||||
describe("resolveAgent", () => {
|
||||
it("returns opencode", () => {
|
||||
const agent = resolveAgent({});
|
||||
expect(agent.name).toBe("opencode");
|
||||
it("returns opencode by default", () => {
|
||||
expect(resolveAgent({}).name).toBe("opencode");
|
||||
});
|
||||
|
||||
it("routes anthropic/* to claude when ANTHROPIC_API_KEY is set", () => {
|
||||
process.env.ANTHROPIC_API_KEY = "sk-test";
|
||||
expect(resolveAgent({ model: "anthropic/claude-opus-4-7" }).name).toBe("claude");
|
||||
});
|
||||
|
||||
it("falls back to opencode for anthropic/* without claude-code creds", () => {
|
||||
expect(resolveAgent({ model: "anthropic/claude-opus-4-7" }).name).toBe("opencode");
|
||||
});
|
||||
|
||||
describe("bedrock routing", () => {
|
||||
it("routes Anthropic Bedrock IDs to claude", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveAgent({ model: "us.anthropic.claude-opus-4-7" }).name).toBe("claude");
|
||||
});
|
||||
|
||||
it("routes Anthropic Bedrock IDs (no region prefix) to claude", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "anthropic.claude-haiku-4-5-20251001-v1:0";
|
||||
expect(resolveAgent({ model: "anthropic.claude-haiku-4-5-20251001-v1:0" }).name).toBe(
|
||||
"claude"
|
||||
);
|
||||
});
|
||||
|
||||
it("routes non-Anthropic Bedrock IDs to opencode", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "amazon.nova-pro-v1:0";
|
||||
expect(resolveAgent({ model: "amazon.nova-pro-v1:0" }).name).toBe("opencode");
|
||||
});
|
||||
|
||||
it("routes Llama IDs to opencode", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "us.meta.llama4-scout-17b-instruct-v1:0";
|
||||
expect(resolveAgent({ model: "us.meta.llama4-scout-17b-instruct-v1:0" }).name).toBe(
|
||||
"opencode"
|
||||
);
|
||||
});
|
||||
|
||||
it("accepts AWS access keys as auth", () => {
|
||||
process.env.AWS_ACCESS_KEY_ID = "AKIA-test";
|
||||
process.env.AWS_SECRET_ACCESS_KEY = "secret-test";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveAgent({ model: "us.anthropic.claude-opus-4-7" }).name).toBe("claude");
|
||||
});
|
||||
|
||||
it("PULLFROG_AGENT override wins over Anthropic auto-routing", () => {
|
||||
process.env.PULLFROG_AGENT = "opencode";
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveAgent({ model: "us.anthropic.claude-opus-4-7" }).name).toBe("opencode");
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveModel", () => {
|
||||
it("PULLFROG_MODEL override wins", () => {
|
||||
process.env.PULLFROG_MODEL = "anthropic/claude-opus";
|
||||
expect(resolveModel({ slug: "openai/gpt" })).toBe("anthropic/claude-opus-4-7");
|
||||
});
|
||||
|
||||
it("PULLFROG_MODEL bypasses bedrock routing entirely", () => {
|
||||
process.env.PULLFROG_MODEL = "openai/gpt";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveModel({ slug: "bedrock/byok" })).toBe("openai/gpt-5.5");
|
||||
});
|
||||
|
||||
it("resolves bedrock/byok to BEDROCK_MODEL_ID", () => {
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveModel({ slug: "bedrock/byok" })).toBe("us.anthropic.claude-opus-4-7");
|
||||
});
|
||||
|
||||
it("throws when bedrock/byok is selected without BEDROCK_MODEL_ID", () => {
|
||||
expect(() => resolveModel({ slug: "bedrock/byok" })).toThrow("BEDROCK_MODEL_ID");
|
||||
});
|
||||
|
||||
it("returns the alias resolve for normal slugs", () => {
|
||||
expect(resolveModel({ slug: "openai/gpt" })).toBe("openai/gpt-5.5");
|
||||
});
|
||||
|
||||
it("returns undefined for no slug + no PULLFROG_MODEL", () => {
|
||||
expect(resolveModel({})).toBeUndefined();
|
||||
});
|
||||
|
||||
// regression: PR #720 review caught that `resolveCliModel("bedrock/byok")`
|
||||
// returns the literal sentinel `"bedrock"` from the alias's `resolve`
|
||||
// field. Without routing-aware handling, PULLFROG_MODEL=bedrock/byok would
|
||||
// leak that sentinel downstream and break agent dispatch.
|
||||
it("PULLFROG_MODEL=bedrock/byok defers to BEDROCK_MODEL_ID, not the sentinel", () => {
|
||||
process.env.PULLFROG_MODEL = "bedrock/byok";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(resolveModel({ slug: "openai/gpt" })).toBe("us.anthropic.claude-opus-4-7");
|
||||
});
|
||||
|
||||
it("PULLFROG_MODEL=bedrock/byok throws if BEDROCK_MODEL_ID is missing", () => {
|
||||
process.env.PULLFROG_MODEL = "bedrock/byok";
|
||||
expect(() => resolveModel({ slug: "openai/gpt" })).toThrow("BEDROCK_MODEL_ID");
|
||||
});
|
||||
});
|
||||
|
||||
+55
-7
@@ -1,6 +1,12 @@
|
||||
import type { Agent } from "../agents/index.ts";
|
||||
import { agents } from "../agents/index.ts";
|
||||
import { getModelProvider, resolveCliModel } from "../models.ts";
|
||||
import {
|
||||
BEDROCK_MODEL_ID_ENV,
|
||||
getModelProvider,
|
||||
isBedrockAnthropicId,
|
||||
resolveCliModel,
|
||||
resolveDisplayAlias,
|
||||
} from "../models.ts";
|
||||
import { log } from "./cli.ts";
|
||||
|
||||
function hasEnvVar(name: string): boolean {
|
||||
@@ -12,6 +18,37 @@ function hasClaudeCodeAuth(): boolean {
|
||||
return hasEnvVar("CLAUDE_CODE_OAUTH_TOKEN") || hasEnvVar("ANTHROPIC_API_KEY");
|
||||
}
|
||||
|
||||
function hasBedrockAuth(): boolean {
|
||||
return (
|
||||
hasEnvVar("AWS_BEARER_TOKEN_BEDROCK") ||
|
||||
(hasEnvVar("AWS_ACCESS_KEY_ID") && hasEnvVar("AWS_SECRET_ACCESS_KEY"))
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* resolve a single slug to its CLI-ready model string. routing aliases
|
||||
* (e.g. `bedrock/byok`) defer to their backing env var instead of the
|
||||
* sentinel stored in `resolve`. shared between PULLFROG_MODEL override
|
||||
* and repo-config slug resolution so both paths get the same routing
|
||||
* semantics — without this helper, `PULLFROG_MODEL=bedrock/byok` would
|
||||
* leak the literal sentinel string `"bedrock"` downstream.
|
||||
*/
|
||||
function resolveSlug(slug: string): string | undefined {
|
||||
const alias = resolveDisplayAlias(slug);
|
||||
if (alias?.routing === "bedrock") {
|
||||
const bedrockId = process.env[BEDROCK_MODEL_ID_ENV]?.trim();
|
||||
if (!bedrockId) {
|
||||
throw new Error(
|
||||
`${BEDROCK_MODEL_ID_ENV} env var is required when the model is set to "${slug}". ` +
|
||||
`set it to an AWS Bedrock model ID (e.g. "us.anthropic.claude-opus-4-7", "amazon.nova-pro-v1:0"). ` +
|
||||
`see https://docs.pullfrog.com/bedrock for setup.`
|
||||
);
|
||||
}
|
||||
return bedrockId;
|
||||
}
|
||||
return resolveCliModel(slug);
|
||||
}
|
||||
|
||||
/**
|
||||
* resolve the effective model for this run.
|
||||
*
|
||||
@@ -19,17 +56,20 @@ function hasClaudeCodeAuth(): boolean {
|
||||
* 1. PULLFROG_MODEL env var — resolved through the alias registry first,
|
||||
* so values like "anthropic/claude-opus" become "anthropic/claude-opus-4-7".
|
||||
* raw specifiers (e.g. "anthropic/claude-opus-4-6") pass through unchanged.
|
||||
* 2. slug from repo config / payload → alias registry
|
||||
* 3. undefined — agent will auto-select
|
||||
* always wins — bypasses Bedrock routing entirely. to test a different
|
||||
* Bedrock model, change `BEDROCK_MODEL_ID`, not `PULLFROG_MODEL`.
|
||||
* 2. slug from repo config / payload → alias registry. routing slugs
|
||||
* (e.g. `bedrock/byok`) defer to a separate env var (`BEDROCK_MODEL_ID`).
|
||||
* 3. undefined — agent will auto-select.
|
||||
*/
|
||||
export function resolveModel(ctx: { slug?: string | undefined }): string | undefined {
|
||||
const envModel = process.env.PULLFROG_MODEL?.trim();
|
||||
if (envModel) {
|
||||
return resolveCliModel(envModel) ?? envModel;
|
||||
return resolveSlug(envModel) ?? envModel;
|
||||
}
|
||||
|
||||
if (ctx.slug) {
|
||||
const resolved = resolveCliModel(ctx.slug);
|
||||
const resolved = resolveSlug(ctx.slug);
|
||||
if (resolved) {
|
||||
return resolved;
|
||||
}
|
||||
@@ -49,7 +89,15 @@ export function resolveAgent(ctx: { model?: string | undefined }): Agent {
|
||||
log.warning(`» unknown PULLFROG_AGENT="${envAgent}" — falling through to auto-select`);
|
||||
}
|
||||
|
||||
// 2. if model is Anthropic and Claude Code credentials are available, use Claude Code
|
||||
// 2. Bedrock routing: when BEDROCK_MODEL_ID is the resolved model, route
|
||||
// Anthropic IDs through claude-code (which supports Bedrock natively
|
||||
// once CLAUDE_CODE_USE_BEDROCK=1) and everything else through opencode's
|
||||
// `amazon-bedrock` provider.
|
||||
if (ctx.model && hasBedrockAuth() && process.env[BEDROCK_MODEL_ID_ENV]?.trim() === ctx.model) {
|
||||
return isBedrockAnthropicId(ctx.model) ? agents.claude : agents.opencode;
|
||||
}
|
||||
|
||||
// 3. if model is Anthropic and Claude Code credentials are available, use Claude Code
|
||||
if (ctx.model) {
|
||||
try {
|
||||
const provider = getModelProvider(ctx.model);
|
||||
@@ -61,6 +109,6 @@ export function resolveAgent(ctx: { model?: string | undefined }): Agent {
|
||||
}
|
||||
}
|
||||
|
||||
// 3. default: OpenCode (universal, supports all providers)
|
||||
// 4. default: OpenCode (universal, supports all providers)
|
||||
return agents.opencode;
|
||||
}
|
||||
|
||||
+84
-2
@@ -9,10 +9,23 @@ const base = {
|
||||
|
||||
const savedEnv = { ...process.env };
|
||||
|
||||
// keys that count as provider auth in `knownApiKeys` and would let the
|
||||
// auto-select path pass without our intent. strip all of them at test setup
|
||||
// so each `it` starts from a clean slate regardless of what's in the dev `.env`.
|
||||
const STRIPPED_PREFIXES_OR_NAMES = [
|
||||
/_API_KEY$/,
|
||||
/^CLAUDE_CODE_OAUTH_TOKEN$/,
|
||||
/^AWS_BEARER_TOKEN_BEDROCK$/,
|
||||
/^AWS_ACCESS_KEY_ID$/,
|
||||
/^AWS_SECRET_ACCESS_KEY$/,
|
||||
/^AWS_SESSION_TOKEN$/,
|
||||
/^AWS_REGION$/,
|
||||
/^BEDROCK_MODEL_ID$/,
|
||||
];
|
||||
|
||||
beforeEach(() => {
|
||||
// strip all known provider keys so tests start clean
|
||||
for (const key of Object.keys(process.env)) {
|
||||
if (key.endsWith("_API_KEY") || key === "CLAUDE_CODE_OAUTH_TOKEN") delete process.env[key];
|
||||
if (STRIPPED_PREFIXES_OR_NAMES.some((re) => re.test(key))) delete process.env[key];
|
||||
}
|
||||
});
|
||||
|
||||
@@ -73,4 +86,73 @@ describe("validateAgentApiKey", () => {
|
||||
expect(() => validateAgentApiKey({ ...base, model: undefined })).toThrow("no API key found");
|
||||
});
|
||||
});
|
||||
|
||||
describe("bedrock routing slug", () => {
|
||||
it("passes with AWS_BEARER_TOKEN_BEDROCK + AWS_REGION + BEDROCK_MODEL_ID", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).not.toThrow();
|
||||
});
|
||||
|
||||
it("passes with AWS access keys + region + model id", () => {
|
||||
process.env.AWS_ACCESS_KEY_ID = "AKIA-test";
|
||||
process.env.AWS_SECRET_ACCESS_KEY = "secret-test";
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "amazon.nova-pro-v1:0";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).not.toThrow();
|
||||
});
|
||||
|
||||
it("throws when BEDROCK_MODEL_ID is missing", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).toThrow(
|
||||
"BEDROCK_MODEL_ID"
|
||||
);
|
||||
});
|
||||
|
||||
it("throws when AWS_REGION is missing", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).toThrow("AWS_REGION");
|
||||
});
|
||||
|
||||
it("throws when no auth is set", () => {
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).toThrow(
|
||||
"AWS_BEARER_TOKEN_BEDROCK"
|
||||
);
|
||||
});
|
||||
|
||||
it("throws when only AWS_ACCESS_KEY_ID is set (missing secret)", () => {
|
||||
process.env.AWS_ACCESS_KEY_ID = "AKIA-test";
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-7";
|
||||
expect(() => validateAgentApiKey({ ...base, model: "bedrock/byok" })).toThrow(
|
||||
"AWS_BEARER_TOKEN_BEDROCK"
|
||||
);
|
||||
});
|
||||
|
||||
// regression: main.ts passes the resolved model into validateAgentApiKey
|
||||
// (`payload.proxyModel ?? resolvedModel ?? payload.model`), which for
|
||||
// bedrock is the raw AWS model ID and has no `/`. parseModel would throw.
|
||||
// see PR #720 e2e run 25821218139 for the original failure mode.
|
||||
it("accepts a raw Bedrock model ID (post-resolveModel) without throwing", () => {
|
||||
process.env.AWS_BEARER_TOKEN_BEDROCK = "bedrock-token";
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-6-v1";
|
||||
expect(() =>
|
||||
validateAgentApiKey({ ...base, model: "us.anthropic.claude-opus-4-6-v1" })
|
||||
).not.toThrow();
|
||||
});
|
||||
|
||||
it("throws on raw Bedrock model ID when AWS auth is missing", () => {
|
||||
process.env.AWS_REGION = "us-east-1";
|
||||
process.env.BEDROCK_MODEL_ID = "us.anthropic.claude-opus-4-6-v1";
|
||||
expect(() =>
|
||||
validateAgentApiKey({ ...base, model: "us.anthropic.claude-opus-4-6-v1" })
|
||||
).toThrow("AWS_BEARER_TOKEN_BEDROCK");
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
+64
-1
@@ -1,4 +1,9 @@
|
||||
import { getModelEnvVars, providers } from "../models.ts";
|
||||
import {
|
||||
BEDROCK_MODEL_ID_ENV,
|
||||
getModelEnvVars,
|
||||
providers,
|
||||
resolveDisplayAlias,
|
||||
} from "../models.ts";
|
||||
import { getApiUrl } from "./apiUrl.ts";
|
||||
|
||||
const knownApiKeys: Set<string> = new Set(Object.values(providers).flatMap((p) => [...p.envVars]));
|
||||
@@ -18,6 +23,26 @@ function buildMissingApiKeyError(params: { owner: string; name: string }): strin
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function buildBedrockSetupError(params: {
|
||||
owner: string;
|
||||
name: string;
|
||||
missing: string[];
|
||||
}): string {
|
||||
const githubSecretsUrl = `https://github.com/${params.owner}/${params.name}/settings/secrets/actions`;
|
||||
|
||||
return `Bedrock model selected but required configuration is missing: ${params.missing.join(", ")}.
|
||||
|
||||
add the missing secret(s) to your GitHub repository at ${githubSecretsUrl}, then reference them in your workflow's \`env:\` block:
|
||||
|
||||
AWS_BEARER_TOKEN_BEDROCK: \${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
|
||||
AWS_REGION: \${{ secrets.AWS_REGION }}
|
||||
${BEDROCK_MODEL_ID_ENV}: \${{ secrets.${BEDROCK_MODEL_ID_ENV} }}
|
||||
|
||||
\`AWS_BEARER_TOKEN_BEDROCK\` may be substituted with \`AWS_ACCESS_KEY_ID\` + \`AWS_SECRET_ACCESS_KEY\` (and optional \`AWS_SESSION_TOKEN\`) if you prefer access keys.
|
||||
|
||||
for full setup instructions, see https://docs.pullfrog.com/bedrock`;
|
||||
}
|
||||
|
||||
function hasEnvVar(name: string): boolean {
|
||||
const value = process.env[name];
|
||||
return typeof value === "string" && value.length > 0;
|
||||
@@ -30,6 +55,22 @@ export function hasProviderKey(model: string): boolean {
|
||||
return requiredVars.some((v) => hasEnvVar(v));
|
||||
}
|
||||
|
||||
function validateBedrockSetup(params: { owner: string; name: string }): void {
|
||||
const hasAuth =
|
||||
hasEnvVar("AWS_BEARER_TOKEN_BEDROCK") ||
|
||||
(hasEnvVar("AWS_ACCESS_KEY_ID") && hasEnvVar("AWS_SECRET_ACCESS_KEY"));
|
||||
|
||||
const missing: string[] = [];
|
||||
if (!hasAuth)
|
||||
missing.push("AWS_BEARER_TOKEN_BEDROCK (or AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY)");
|
||||
if (!hasEnvVar("AWS_REGION")) missing.push("AWS_REGION");
|
||||
if (!hasEnvVar(BEDROCK_MODEL_ID_ENV)) missing.push(BEDROCK_MODEL_ID_ENV);
|
||||
|
||||
if (missing.length > 0) {
|
||||
throw new Error(buildBedrockSetupError({ owner: params.owner, name: params.name, missing }));
|
||||
}
|
||||
}
|
||||
|
||||
export function validateAgentApiKey(params: {
|
||||
agent: { name: string };
|
||||
model: string | undefined;
|
||||
@@ -38,6 +79,28 @@ export function validateAgentApiKey(params: {
|
||||
}): void {
|
||||
// if a specific model is configured, only check that model's required env vars
|
||||
if (params.model) {
|
||||
// routing slugs (e.g. bedrock) get a tailored validation path because
|
||||
// their auth shape doesn't match the standard "any one envVar present"
|
||||
// rule (Bedrock needs auth + region + model-id, with auth being either
|
||||
// a bearer token OR an access-key pair).
|
||||
const alias = resolveDisplayAlias(params.model);
|
||||
if (alias?.routing === "bedrock") {
|
||||
validateBedrockSetup({ owner: params.owner, name: params.name });
|
||||
return;
|
||||
}
|
||||
|
||||
// upstream `resolveModel` translates `bedrock/byok` into the raw Bedrock
|
||||
// model ID (e.g. `us.anthropic.claude-opus-4-6-v1`), which has no `/`
|
||||
// and so isn't parseable as `provider/model`. these IDs only reach this
|
||||
// function via routing aliases, so re-run the bedrock setup check rather
|
||||
// than falling through to `getModelEnvVars` (which would throw inside
|
||||
// parseModel). resolveModel itself already enforced BEDROCK_MODEL_ID,
|
||||
// but auth + region are still validated here.
|
||||
if (!params.model.includes("/")) {
|
||||
validateBedrockSetup({ owner: params.owner, name: params.name });
|
||||
return;
|
||||
}
|
||||
|
||||
const requiredVars = getModelEnvVars(params.model);
|
||||
// free models have no required env vars — skip validation entirely
|
||||
if (requiredVars.length === 0) return;
|
||||
|
||||
Reference in New Issue
Block a user