Skip to content

Validate command silently skips all validators when given a RecipeMetadata overlay #526

Description

@sara4dev

Summary

When aicr validate --recipe receives a RecipeMetadata overlay file (instead of a hydrated RecipeResult), all validators are silently skipped (selected=0) with no error or warning.

Root Cause

The validation section in a RecipeMetadata overlay is nested under spec::

kind: RecipeMetadata
spec:
  validation:
    deployment:
      checks: [...]
    conformance:
      checks: [...]

But the validator expects it at the top level (RecipeResult format):

kind: RecipeResult
validation:
  deployment:
    checks: [...]
  conformance:
    checks: [...]

When the overlay is passed directly, the validation config is never found, and all phases report selected=0 with no checks executed.

Reproduction

# This works (hydrated recipe):
aicr recipe --service eks --accelerator gb200 --intent inference -o recipe.yaml
aicr validate --recipe recipe.yaml   # runs all checks

# This silently skips everything (overlay file):
aicr validate --recipe recipes/overlays/gb200-eks-ubuntu-inference-dynamo.yaml
# Output: selected=0 for all phases, no checks run, no error

Expected Behavior

When given a RecipeMetadata overlay, aicr validate should either:

  1. Detect the input kind and resolve overlays through the recipe builder before running validation, or
  2. Return a clear error indicating that a hydrated RecipeResult is required

Metadata

Metadata

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions