Skip to content

Commit verified reasoning and input-modality values to the USAi catalog #541

Closed

Description

@mogul

After a recorded live verification, commit verified reasoning / inputModalities values to the USAi catalog and shipped snapshots, with sources.capabilitiesVerified set to <YYYY-MM-DD>; opencode <version>. Models that fail or were not tested stay absent (unknown). Never render text-only by default.

Blocked on: schema adoption by consumers, and the live image verification run, both tracked as follow-ups to #366. Trigger: both are complete.

AI-assisted tracker entry (OpenCode). Human review required.

Activity

  1. mogul commented on Oct 25, 2026

    @mogul
    ContributorAuthor

    Design note from review: sources is rebuilt on every build-catalog run and the builder never emits capabilitiesVerified, so a hand-added verification record and any committed capability values are erased by a plain regeneration (the guard test then fails, which is the safe direction). This issue must decide how verified capabilities survive regeneration, for example a reviewed verified-capabilities input file that the builder merges, and update the guard test to match.

  2. mogul commented on Nov 5, 2026

    @mogul
    ContributorAuthor

    🤖: AI agent (First Mate), posting on behalf of @mogul.

    Closing as delivered by #575 (merged 2026-11-05, released in Patterns v1.19.0).

    • Live run: OpenCode 2.0.30 against the USAi gateway, 3 samples per configuration, 24 configuration facts and 189 stored replies including 63 negative-control replies (0 token leaks). Evidence is committed at integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30.md with the results JSON/TSV and replies next to it.
    • Published: inputModalities ["text","image"] for 12 models. 9 candidates misread the image and are not published; 3 were unavailable (two Gemini models returned 403 model_blocked_globally, one Llama model returned 400 for no streaming tool use).
    • reasoning stays absent for all models (ADR 0006 amended in docs(opencode): add reasoning capability validation procedure #577; follow-up stays in Verify and publish reasoning capability metadata for USAi models #574).
    • The owner accepted the audio, video and PDF narrowing for the 12 models, as ADR 0006 requires.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions