feat(usai-catalog): refresh models and withhold undeployed gateway entries - #519

Summary

USAi released new models. This regenerates the harness-neutral catalog and both downstream kit artifacts from the live gateway and models.dev feeds, and withholds three models the gateway advertises but cannot serve.

Nothing here was hand-edited. Every artifact is generated output.

Model delta

The gateway now lists 26 models against the committed catalog's 20.

Added and shipped

• claude-sonnet-5.5
• gpt-6-astra

Retired upstream

• gemini-2.5-pro — no longer in /v1/models

Already filtered by the existing non-chat pattern

• cohere_english_v3, text-embedding-005 — embedding models

Final shipped catalog: 21 chat models.

Three advertised GPT-6 models are withheld

The gateway lists these in /v1/models, but a /chat/completions call to each returns HTTP 404:

{"detail":"404: The API deployment for this resource does not exist. …"}

Each was confirmed with a real completion request and re-probed to rule out a transient. Against a control group on the same route, the failure is specific rather than environmental:

Shipping them would put a model in the picker that fails the moment a user selects it. They are suppressed through the existing non-chat exclusion map in both the catalog builder and the kit generator, each carrying the measured reason, so the mechanism is not new and the rationale travels with the code. Remove the entries once the deployments land.

This is a gateway-side inconsistency, not a defect in this repo. Reporting it to the USAi team separately so the model list and the deployments can be reconciled at the source.

Snapshot subset correction

model-catalog-snapshot.json is a hand-trimmed subset with one model per vendor, guarded by a byte-equality test against the normalizer. It kept gemini-2.5-pro, which no longer exists upstream, so that test failed.

It moves to gemini-2.5-flash, the surviving model on that line, preserving the subset's vendor coverage. The subset is deliberately small; this is the minimum change that keeps the gate meaningful rather than dropping a vendor.

prime-agent lockstep

prime-agent/files/home/prime-agent-config/models.json is generated from the same catalog.json by the prime-agent emitter. Its byte-exact lockstep test correctly caught the committed file as stale once the catalog moved, so it is regenerated from the emitter rather than edited.

That test earning its keep here is worth noting: without it, the prime-agent kit would have silently shipped a stale model list.

One behavioral note

gpt-6-astra rejects max_tokens below 16:

{"detail":"integer below minimum value. Expected a value >= 16, but got 8 instead."}

Other models accept 8. A reasoning-model floor. Harmless with the shipped config, which sets no such cap, but it will surface for anyone who sets a very small limit. Not worked around here, since the right fix belongs to whatever sets the cap.

Verification

• build-catalog.mjs --check — up to date, 21 models
• usai-provider tests — 76/76
• providers/usai tests — 100/100, including both byte-exact emitter guards
• validate-kits.py --strict — 10 kits plus 2 templates valid
• generate_index.py --check — INDEX current
• git diff --check — clean

Beyond the suites, the shipped JSONC was parsed through the real startup merge path (parseJsonc from merge-global-config.mjs), confirming:

• 21 models parse
• both working new models present
• all three withheld models, gemini-2.5-pro, and both embedding models absent
• model and small_model defaults still resolve to models that exist

Default selections are unchanged: usai/claude-sonnet-5 and usai/claude_4_5_haiku.

make validate fails 4 validators (frontmatter schema, cross-reference, network-tier baseline, balanced-allowlist guard). I verified by stashing that all four fail identically on unmodified origin/main — pre-existing and unrelated to this change. Not introduced here, and not fixed here.

Downstream

Kits are consumed at a pinned ref, so this does not reach users until a patterns release is cut and GSA-TTS/agentic-coding-quickstart bumps PATTERNS_KIT_REF. That pin currently sits two releases behind, which is a separate problem already being addressed.

Type of Change

• New feature (non-breaking change that adds functionality)

Safety Checklist

• No secrets, API keys, or tokens. The catalog's apiKeyEnv holds the env-var name, never a value; a test asserts no key material appears in any emitted artifact.
• No PII, CUI, customer data, or internal URLs
• Validation run before submitting
• Generated artifacts regenerated, not hand-edited

AI Assistance Disclosure

• AI tools were used in this contribution (OpenCode).
• I verified all AI-generated content, including every model probe above.