[BUG]: nv_build with no llm-model returns HTTP 404 for our NVIDIA Build key, so RUBRIC_EVAL and the LLM security verification do not run #510

What happened?

Thank you for the reusable workflow and the nv_build provider; with a model named, the LLM checks ran well for us.

ci/README.md at v0.9.0 says llm-model is optional. With llm-provider: nv_build and llm-model left empty, the run uses the default from src/skillevaluator/provider_config.py (CHAT_DEFAULT_NVIDIA = "nvidia/nemotron-3-super-120b-a12b", line 52 at v0.9.0). For our NVIDIA Build key that model answers HTTP 404, so RUBRIC_EVAL fails without running ("LLM judge unavailable; rubric evaluation did not run") and the security scan lists llm-verification as incomplete. The message the run printed was: "Model or endpoint unavailable. Check SKILL_EVAL_LLM_MODEL against the provider's available models."

Expected: the default nv_build model is callable with an NVIDIA Build key, or the run says the default model itself is unavailable and suggests setting llm-model.

Actual: a 404 that reads like a setup mistake on our side, with no model named by us.

What we checked outside the workflow, with the same account's key, one 4-token call per model on https://integrate.api.nvidia.com/v1:

• GET /models lists nvidia/nemotron-3-super-120b-a12b.
• POST /chat/completions with that model: HTTP 404, "Function '…': Not found for account '…'" (ids left out here).
• nvidia/nemotron-3-ultra-550b-a55b, nvidia/nemotron-3.5-lightning-30b-a3b and nvidia/nemotron-3-nano-omni-30b-a3b-reasoning: HTTP 200.

So the model is listed but not callable on our account. We have one account to test with and cannot tell whether this is specific to it or to free NVIDIA Build accounts in general.

Possible directions, offered, not prescribed: a default model that NVIDIA Build serves to every key; or, on a 404 from the default model, a message that names the default and suggests llm-model; or a line in ci/README.md that the default may not be callable on every account.

Reproduction steps

1. In a public repository with one skill under skills/, add ci/bundles/0.9.0/skill-checks-llm.yml from the v0.9.0 tag with llm-provider: nv_build and llm-model empty, and an NVIDIA Build key in the secret LLM_API_KEY.
2. Run it. RUBRIC_EVAL fails with the 404 above.
3. Set llm-model: nvidia/nemotron-3-ultra-550b-a55b and run again: the rubric and the LLM verification run.

Our runs: https://github.com/TheColliery/CoalFace/actions/runs/43437857577 (empty llm-model, 404) and https://github.com/TheColliery/CoalFace/actions/runs/43440953188 (llm-model set, both LLM checks ran).

SkillEvaluator version or commit

v0.9.0 (reusable workflow ba7779ad617c6444c98b6241a75bce92067e9d56, Tier 1 image sha256:e96e3aaad89bf78c4f3f8c621e88005d18bd7c485b29c0d7945864700e5944e6)

Environment

GitHub-hosted runner through the reusable workflow; provider nv_build; a personal NVIDIA Build API key.

Before submitting

• I agree to follow this project's Code of Conduct
• I have searched the open issues and found no duplicate of this report