Skip to content

Sync metric docstrings with the keys actually returned#86

Open
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:docs/metrics-docstrings-match-returns
Open

Sync metric docstrings with the keys actually returned#86
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:docs/metrics-docstrings-match-returns

Conversation

@lonexreb
Copy link
Copy Markdown
Contributor

@lonexreb lonexreb commented May 4, 2026

Problem

The docstrings in src/alpamayo_r1/metrics/metric_utils.py and src/alpamayo_r1/metrics/distance_metrics.py advertise return-dict keys that are never produced. Anyone iterating these dicts looking for the documented _sq keys gets a KeyError.

1. summarize_metric (metric_utils.py)

Docstring claims it returns:

name: shape [B] # average
name_sq: shape [B] # average square
name_std: shape [B] # standard deviation over N

But the body only adds name_std:

if val.shape[1] > 1 and not disable_summary:
    mean_sq = val.pow(2).mean(1)
    std = (mean_sq - mean**2).sqrt()
    result[metric_name + "_std"] = std

mean_sq is computed as an intermediate of std and discarded. Also, the gating on N > 1 is not obvious from the docstring.

2. compute_minade (distance_metrics.py)

Docstring lists min_ade_sq, but the body returns min_ade plus per-horizon keys of the form min_ade/by_t={t * time_step:.1f} (then optionally _std variants via summarize_metric). The per-horizon keys are the function's headline output and were entirely undocumented.

3. compute_grouped_corner_distance (distance_metrics.py)

Docstring lists corner_distance_sq, but the body returns corner_distance (and corner_distance_std from summarize_metric when N > 1 and disable_summary=False).

Fix

Pure docstring change — no code logic touched. Aligns each docstring with the actual returned keys and documents how summarize_metric layers _std on top.

Verification

Skimmed the call sites (none rely on the documented _sq keys), and the new wording matches the runtime behavior verifiable by reading the function bodies.

The docstrings in src/alpamayo_r1/metrics/metric_utils.py and
src/alpamayo_r1/metrics/distance_metrics.py advertise return-dict keys
that are never produced:

- summarize_metric() docstring lists `name_sq`, but the body only adds
  `name_std` (mean_sq is computed as an intermediate of std and not
  stored). Also clarifies that `_std` is added only when N > 1, which
  is not obvious from the current wording.

- compute_minade() docstring lists `min_ade_sq`, but the body returns
  `min_ade` plus per-horizon keys of the form
  `min_ade/by_t={t * time_step:.1f}` (and corresponding `_std` keys
  via summarize_metric). The per-horizon keys are the headline output
  of the function and were undocumented.

- compute_grouped_corner_distance() docstring lists
  `corner_distance_sq`, but the body returns `corner_distance` (and
  `corner_distance_std` from summarize_metric when N > 1).

This commit aligns the docstrings with the actual returned keys and
documents how summarize_metric layers `_std` on top. Pure docs change.

Signed-off-by: lonexreb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant