Commit Graph

2 Commits

Author SHA1 Message Date
Colin McDonnell c608051b79 sanitize mcp schemas for Gemini; fix gpt-codex-mini alias; add matrix filter
Gemini's generateContent API rejects arktype's `{anyOf:[{enum:[...]}]}` string-enum
encoding, `$schema` metadata, and `anyOf` with sibling fields. Port the old
sanitizer back as an isolated module (action/mcp/geminiSanitizer.ts) and gate it
on `isGeminiRouted(ctx)` so non-gemini routes see the original schema. Wires
`resolvedModel` onto ToolContext so the sanitizer can see the upstream specifier.

Also bumps `openai/gpt-codex-mini` alias from the deprecated `codex-mini-latest`
to `gpt-5.1-codex-mini`, matching the openrouter resolve.

Adds a `filter` workflow_dispatch input + MATRIX_FILTER env that restricts the
models-live matrix to aliases matching a substring, so we can iterate on a
single provider (e.g. `filter=gemini`) without paying to run every model.
2026-04-16 23:09:32 +00:00
Colin McDonnell 5e6ff67623 move models.dev drift tests to main-only; add per-alias live smoke matrix
PR CI kept breaking on upstream catalog drift (new model ships on models.dev,
OpenRouter renames an id, etc.) — failures unrelated to the PR's contents.
split the model-alias test suite so PRs only see pure-logic checks, and push
the external-state drift + end-to-end coverage to main.

test organization:
- action/test/models.test.ts keeps pure invariants: openRouterResolve
  completeness and fallback-chain resolution. runs on every PR.
- action/test/models-catalog.main.test.ts gets the 4 network-dependent
  describes (models.dev validity x2, OpenRouter API validity, latest-model
  snapshot). runs only on main push via a dedicated vitest config
  (vitest.main.config.ts + `pnpm test:catalog`).

new CI jobs in .github/workflows/test.yml:
- models-catalog: `pnpm test:catalog` on every main push. detects upstream
  catalog drift so we can react at the next convenient window.
- models-live: 38-entry matrix that invokes the agent harness end-to-end
  against the real provider for each alias in models.ts. generated from
  action/test/list-aliases.ts. runs only on main push AND only when
  resolution-affecting files changed (action/models.ts, action/package.json,
  action/agents/**) — the exact shape of the opus 4.7 incident.

test/run.ts: PULLFROG_MODEL now flows through from process.env so the live
matrix can pin an alias per job without the per-agent default clobbering it.

Made-with: Cursor
2026-04-16 21:10:15 +00:00