Skip to content

fix: GPU SKU classifier substring match silently mismaps SKUs #1349

Description

@mchmarny

Summary

The GPU SKU classifier (ParseGPUSKU) identifies a node's accelerator by substring-matching nvidia-smi's ProductName against a pattern table. Because it uses strings.Contains, any SKU whose product name contains a shorter registered pattern is silently collapsed onto the wrong accelerator. This defeats the classifier's own unknown-sku safety signal and produces fingerprints that misrepresent the hardware.

Current behavior / evidence

On an Oracle OKE cluster with nvidia.com/gpu.product=NVIDIA-L40S, the snapshot fingerprint records:

fingerprint:
  accelerator:
    source: nodeTopology.label.nvidia.com/gpu.product
    value: l40        # the trailing "S" is dropped

NVIDIA-L40S contains the substring L40, so it matches {"L40", "l40"} and is reported as a (valid-but-wrong) l40 instead of being flagged unknown-sku.

This is a class of bug, not a one-off. The table already carries two hand-patched workarounds for the same root cause:

  • {"GB200", ...} is ordered before {"B200", ...} so a Grace-Blackwell node isn't read as b200.
  • {"GH200", ""} is an explicit exclusion so Grace-Hopper isn't read as h200.

Each new collision currently requires another manual ordering/exclusion entry. Known/likely live collisions that are not yet patched:

  • NVIDIA RTX A1000"A1000" contains "A100" → silently fingerprints as datacenter a100.
  • NVIDIA L40Sl40 (the reported case).

Proposed approach

Replace substring matching with token-boundary matching in ParseGPUSKU: tokenize the product name (split on whitespace and -) and match whole tokens, with contiguous-token-sequence matching for multi-word patterns (RTX PRO 6000). Unrecognized tokens fall through to unknown-sku (the honest signal).

This structurally eliminates the entire collision class and removes the need for per-SKU ordering/exclusion hacks:

  • NVIDIA L40S → token L40S (not L40) → no false match → unknown-sku.
  • NVIDIA RTX A1000 → token A1000 (not A100) → no false a100.
  • NVIDIA A100-SXM4-40GB → token A100a100. ✓
  • NVIDIA GB200 NVL72 → token GB200gb200; B200 is no longer a substring concern, so the GB200-before-B200 ordering hack and the GH200 exclusion become unnecessary.

Acceptance criteria

  • ParseGPUSKU matches on token boundaries (handles space and - delimiters; supports the multi-word RTX PRO 6000 pattern).
  • Unrecognized product strings return "" so the fingerprint records unknown-sku (raw model already preserved in measurements).
  • Ordering/exclusion workarounds removed where token matching makes them redundant (GB200/B200, GH200), with tests proving the behavior still holds.
  • Test matrix asserts decoys do not false-map: RTX A1000, L40S, L4, A800, H800, GH200, plus all currently supported SKUs continue to resolve correctly.

Notes / related

  • Making l40s (or any newly distinguished SKU) a first-class, selectable accelerator is a separate, larger change (criteria enum, OpenAPI, CLI/API, docs) and should land together with actual recipe coverage for that SKU — otherwise selecting it just yields "no matching recipe." This issue is only about fixing the misclassification, not adding new enum values.

Metadata

Metadata

Assignees

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions