Colin McDonnell
8f9208bd3f
* add Amazon Bedrock as a routing slug introduces a single `bedrock/byok` catalog entry that the harness translates to the appropriate Bedrock model ID at run time via `BEDROCK_MODEL_ID`. routes Anthropic IDs through claude-code (with `CLAUDE_CODE_USE_BEDROCK=1`) and everything else through opencode's `amazon-bedrock` provider — keeps the catalog flat for an audience that needs version pinning rather than aliasing. accepts either `AWS_BEARER_TOKEN_BEDROCK` or `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` for auth; both validated alongside `AWS_REGION` and `BEDROCK_MODEL_ID` in `validateAgentApiKey`. catalog drift tests, the bumps cron, and per-alias smoke scripts all skip routing slugs since there's no fixed `resolve` to validate. docs/bedrock.mdx walks through setup; wiki/model-resolution.md has a section explaining why bedrock breaks the usual alias pattern. closes pullfrog/pullfrog#40 * ci: add bedrock env vars to test workflows mirrors the new bedrock provider's required env vars (AWS_BEARER_TOKEN_BEDROCK inherited from org secret + AWS_REGION + BEDROCK_MODEL_ID hardcoded) into both .github/workflows/test.yml files so the ci.test "env vars cover all provider API keys" assertion passes. * docs(bedrock): clearer setup flow + screenshot of model selector restructures the setup section into three concrete steps in execution order: select Bedrock from the dropdown, store the bearer token as a secret (Pullfrog or GitHub — links to keys.mdx for the trade-off), then add region + model id directly in pullfrog.yml since neither is sensitive. enable-model-access in the Bedrock console moved to step 4 (only required once per model and only when AWS rejects the call, not blocking on first run). adds a screenshot of the console model selector with Amazon Bedrock selected so readers can recognize the UI state they're aiming for. * fix(bedrock): tolerate raw Bedrock model IDs in validateAgentApiKey main.ts passes the resolved model into validateAgentApiKey (`payload.proxyModel ?? resolvedModel ?? payload.model`). For Bedrock, `resolveModel` translates `bedrock/byok` into the raw AWS model ID (e.g. `us.anthropic.claude-opus-4-6-v1`), which has no `/` and so trips parseModel inside getModelEnvVars. Detect the no-slash case and re-run the bedrock setup check (auth + region; BEDROCK_MODEL_ID is already enforced upstream by resolveModel). Caught by PR #720 e2e dispatch on pullfrog/preview-720-bedrock — "invalid model slug 'us.anthropic.claude-opus-4-6-v1' — expected 'provider/model'". Two regression tests cover the raw-ID path. * fix(bedrock): always prepend amazon-bedrock/ prefix when bedrock-routed opencode.ts was gating the prefix-injection on `!isBedrockAnthropicId(rawModel)`, on the theory that Anthropic Bedrock IDs always go through claude-code. But `PULLFROG_AGENT=opencode` is a documented escape hatch — when it forces opencode for an Anthropic Bedrock model, the prefix still has to be added or opencode fails with 'Model not found: <modelId>/.'. The Anthropic-vs-other discriminant only belongs in resolveAgent. Once an agent is selected, it should consistently honor the bedrock route. Caught by the PULLFROG_AGENT=opencode + Opus 4.6 e2e on pullfrog/preview-720-bedrock — run 25823437606. * ui+docs(bedrock): bespoke setup callout + clearer docs UI: - BedrockSetupCallout in components/AgentSettings.tsx covers both the Model costs section and the onboarding card. Detects bedrock via resolveDisplayAlias().routing === "bedrock", shows a dedicated message ("store AWS_BEARER_TOKEN_BEDROCK as a secret, then put AWS_REGION + BEDROCK_MODEL_ID directly in pullfrog.yml") + link to the setup guide. Replaces the generic "X, Y, or Z is required" prompt that misrepresented the three values as three separate secrets to add (and used the wrong "or" connector for what's actually an AND). - OnboardingCard re-uses the same callout with the gradient-card variant. Docs: - Drop the obsolete "Enable model access" step. AWS retired the manual enrollment page; foundation models auto-enable on first invocation. Anthropic models still need a one-time use-case form for first-time users — surfaced under the AccessDenied troubleshooting entry. - Drop the "Testing a different model in one run" PULLFROG_MODEL note. It introduced the secrets-vs-vars distinction we want to keep out of the bedrock setup story. - Step 3 already recommends hardcoding region + model id in pullfrog.yml. Workflow template: - The default pullfrog.yml customers receive (utils/github/pullfrog.yml.ts) now references AWS_BEARER_TOKEN_BEDROCK from secrets but inlines AWS_REGION and BEDROCK_MODEL_ID as plain values. Matches the docs. * fix(bedrock): three review-caught edges in routing + UI copy Addresses three real issues from PR #720 review: 1. agent.ts: PULLFROG_MODEL=bedrock/byok no longer leaks the literal sentinel "bedrock" downstream. resolveCliModel returns the alias's resolve field verbatim, which for routing entries IS the sentinel. Refactored both the env-override and slug-lookup paths through a shared resolveSlug() that recognizes routing aliases and defers to their backing env var (BEDROCK_MODEL_ID). 2. models.ts: isBedrockAnthropicId() now anchors on a discrete dot/slash/colon-segment match (case-insensitive) instead of a substring contains. The substring check was fragile in both directions for inference-profile ARNs (BEDROCK_MODEL_ID accepts ARNs per AWS docs) — a non-Anthropic profile whose user-chosen name contained "anthropic" would mis-route to claude-code, and an Anthropic profile whose name omitted it would miss CLAUDE_CODE_USE_BEDROCK=1. 3. AgentSettings.tsx: BedrockSetupCallout's configured-state copy showed "AWS_BEARER_TOKEN_BEDROCK configured" even when the user satisfied the gate via AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY, gaslighting access-key users about a secret they never set. Detect which auth method is actually present and name the right secret(s) in the success message. Regression tests in models.test.ts (5 new isBedrockAnthropicId cases including positive and negative ARN forms) and agent.test.ts (2 new PULLFROG_MODEL=bedrock/byok cases). 171/171 action tests pass. * yml template: add commented AWS access-key alternative for Bedrock auth Mirrors the IAM access-key path verified end-to-end on PR #720 e2e run 25830764987. Bearer token stays as the primary nudge; the access-key pair is the fallback for users who can't mint Bedrock API keys. * yml template: drop redundant 'or, alternatively' annotation * ui+docs(bedrock): rewrite callout copy + refresh screenshot Reframes the BedrockSetupCallout away from generic BYOK language to a Bedrock-specific message: leads with "Amazon Bedrock is configured entirely via environment variables", lists all four (auth, region, model id), and ends with the requested CTA sentence ("click below to learn more about Bedrock support in Pullfrog"). Promotes the "Bedrock setup guide" docs link from an inline anchor to a prominent button (always visible, regardless of auth state). The "Add AWS_BEARER_TOKEN_BEDROCK" affordance is now a secondary chip shown only when no auth secret is configured. Refreshes docs/images/model-selector-bedrock.png to capture the new callout — the prior screenshot still showed the old generic "BYOK / X, Y, or Z required" wording.
105 lines
2.8 KiB
YAML
105 lines
2.8 KiB
YAML
name: Test
|
|
|
|
on:
|
|
pull_request:
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- uses: pnpm/action-setup@v4
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version: "24"
|
|
cache: "pnpm"
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm typecheck
|
|
- run: pnpm test
|
|
|
|
agents:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
strategy:
|
|
fail-fast: true
|
|
matrix:
|
|
agent: [claude, opencode]
|
|
test:
|
|
[
|
|
mcpmerge,
|
|
nobash,
|
|
restricted,
|
|
skill-invoke-claude,
|
|
skill-invoke-opencode,
|
|
smoke,
|
|
token-exfil,
|
|
]
|
|
exclude:
|
|
- agent: claude
|
|
test: skill-invoke-opencode
|
|
- agent: opencode
|
|
test: skill-invoke-claude
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
|
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}
|
|
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
|
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
|
|
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
|
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|
|
MOONSHOT_API_KEY: ${{ secrets.MOONSHOT_API_KEY }}
|
|
OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
|
|
AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
|
|
AWS_REGION: us-east-1
|
|
BEDROCK_MODEL_ID: us.anthropic.claude-opus-4-6-v1
|
|
PULLFROG_MODEL: ${{ vars.PULLFROG_MODEL }}
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- uses: pnpm/action-setup@v4
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version: "24"
|
|
cache: "pnpm"
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm runtest ${{ matrix.test }} ${{ matrix.agent }}
|
|
|
|
agnostic:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
strategy:
|
|
fail-fast: true
|
|
matrix:
|
|
test:
|
|
[
|
|
git-permissions,
|
|
githooks,
|
|
pkg-json-scripts,
|
|
push-disabled,
|
|
push-enabled,
|
|
push-restricted,
|
|
timeout,
|
|
]
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- uses: pnpm/action-setup@v4
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version: "24"
|
|
cache: "pnpm"
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm runtest ${{ matrix.test }}
|