Skip to content

Use provider-defined USAi facts when seeding SBX backend verification #552

Closed

Description

@basiliskus

Defect

The SBX verifier's scope-aware seed guard and seed command hard-code the fallback USAi host and key environment variable:

# scripts/verify-backends:503-511
if [ "$backend" = "sbx" ]; then
  if ! sbx secret ls 2>/dev/null | awk '
       $1 == "(global)" {
         for (i = 2; i <= NF; i++) if ($i == "USAI_API_KEY") { found = 1; exit }
       }
       END { exit(found ? 0 : 1) }
     '; then
    sbx secret set-custom --host "$USAI_HOST" --env USAI_API_KEY \
      --value "$VERIFY_USAI_DUMMY" >/dev/null 2>&1 || true
  fi
fi

The hard-coded row match is at scripts/verify-backends:506, and the hard-coded seed is at scripts/verify-backends:510. The script contains no reference to USAI_PROVIDER_KEY_ENV.

Step 3 then calls acq secret has -g usai at scripts/verify-backends:519. That command loads provider facts at acq:1611-1614 and delegates to acq_backend_key_present; SBX checks the proxy row named by USAI_PROVIDER_KEY_ENV at acq.backends/sbx.sh:1717-1724.

Impact

If provider facts define a key name other than USAI_API_KEY, verifier step 2 and production step 3 inspect different rows. A pre-existing fallback row can make the verifier guard succeed, so it does not create the provider-defined row. Production then checks the provider-defined name, reports the key non-injectable, and the verifier skips the SBX backend and agent-install sections.

Skipped checks do not affect the final result: scripts/verify-backends:1388-1393 reports SKIP but exits based only on FAIL. The command can therefore exit zero without exercising the SBX behavior it was invoked to verify.

Reproduction

The probe below extracts the released source directly from commit af9814d802cf798850c56c4d8715e9bb376802f9. It models provider facts selecting CUSTOM_API_KEY while the SBX table contains only the fallback row. The first check is the verifier's guard; the second is the production predicate used by step 3.

scratch=$(mktemp -d "${TMPDIR:-/tmp}/paseo-review-main.XXXXXX")
git archive af9814d802cf798850c56c4d8715e9bb376802f9 | tar -x -C "$scratch"
SCRATCH="$scratch" bash -c '
  . "$SCRATCH/acq.backends/common.sh"
  . "$SCRATCH/acq.backends/sbx.sh"
  USAI_PROVIDER_KEY_ENV=CUSTOM_API_KEY
  listing="CUSTOM SECRETS
SCOPE TARGETS ENV PLACEHOLDER
(global) custom.example.gov USAI_API_KEY old-placeholder"
  sbx() {
    [ "$1 $2" = "secret ls" ] && printf "%s\n" "$listing"
  }
  printf "%s\n" "$listing" | awk '\''
    $1 == "(global)" {
      for (i = 2; i <= NF; i++) {
        if ($i == "USAI_API_KEY") { found = 1; exit }
      }
    }
    END { exit(found ? 0 : 1) }
  '\''
  guard_rc=$?
  acq_backend_key_present usai ""
  check_rc=$?
  printf "provider_key=%s verifier_guard_rc=%s production_check_rc=%s\n" \
    "$USAI_PROVIDER_KEY_ENV" "$guard_rc" "$check_rc"
'

Observed output:

provider_key=CUSTOM_API_KEY verifier_guard_rc=0 production_check_rc=1

The verifier believes the required global row already exists, while the production predicate correctly reports that the provider-defined row is absent.

Severity

Major, currently latent. The v1.12.0 usai-provider kit pinned by v4.0.0 does not contain provider-facts/usai.env, so current runs use the fallback USAI_API_KEY and both sides happen to agree. The mismatch becomes live when both conditions hold:

  1. the provider-facts assignment-loss defect is fixed so loaded values survive; and
  2. a valid provider artifact defines a key environment name other than USAI_API_KEY.

The OpenCode agent kit becoming live in v4.0.0 does not change this latency condition; provider facts are loaded from the separate usai-provider kit.

Fix requirement

Make scripts/verify-backends derive the SBX row match, set-custom --host, and set-custom --env values from the same loaded provider facts used by acq secret has. The verifier must fail rather than count the backend as skipped when it cannot establish a required injectable USAi binding for an installed backend. Add a regression test with a non-default provider key environment name that proves seeding and the production presence check agree.

This defect shipped in v4.0.0, published at 2026-10-06 00:44:00Z, with target commit af9814d802cf798850c56c4d8715e9bb376802f9.

Related: #481 tracks the provider-facts split feature; this issue is the verifier mismatch exposed by that contract. #547 is a different false-green defect in the same verifier involving an ignored run_acq return code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions