Skip to content

Sync DistanceMetrics docstrings with the actual signature and returns#93

Open
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:docs/distance-metrics-evaluate-docstring
Open

Sync DistanceMetrics docstrings with the actual signature and returns#93
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:docs/distance-metrics-evaluate-docstring

Conversation

@lonexreb
Copy link
Copy Markdown
Contributor

@lonexreb lonexreb commented May 4, 2026

Problem

Two docstring drift bugs in DistanceMetrics (src/alpamayo_r1/metrics/metric_api.py) — same pattern that #86 already fixed for the lower-level functions in distance_metrics.py / metric_utils.py.

1. Phantom group_by_scenarios parameter on __init__

def __init__(self, prefix: str = "", time_step: float = 0.1):
    """Args:
    prefix (str, optional): ...
    group_by_scenarios (list[str], optional): Defaults to None.
        if not None, then the metrics will be grouped by the scenario names in this list.
    """

The signature has only prefix and time_step. group_by_scenarios is documented but does not exist — likely a leftover from an earlier API. Worse, time_step (which is the actual second parameter and matters for the per-horizon keys) is undocumented.

2. _sq keys in evaluate() Returns block that are never returned

Returns dict[str,Tensor]:
    min_ade: [B] average min_ade ...
    corner_distance: [B] ...
    # if N > 1, we have the following extra data item:
    # for each value above, we collect the statistics _sq and _std, for example:
    min_ade_sq: [B] average min_ade^2 ...
    min_ade_std: [B] std of min_ade

The body only emits min_ade, min_ade/by_t={H:.1f}, ade, ade/by_t=3.0, corner_distance, plus _std variants from summarize_metric when num_traj_sets > 1. The _sq keys exist nowhere; the ade* and by_t* keys (which are the bulk of the actual output) are completely undocumented.

Fix

Pure docstring change. No code logic touched.

  • Replace the dead group_by_scenarios entry with an accurate description of time_step.
  • Rewrite the Returns block to list every key the body actually emits, mirror the format used in Sync metric docstrings with the keys actually returned #86 for compute_minade, and call out that all keys are prefixed via apply_prefix(self.prefix, ...).

Verification

The new key list matches the body's metric_dict.update(...) calls (around lines 232 for ade, 241 for ade/by_t=3.0, and the return of compute_grouped_corner_distance for corner_distance).

src/alpamayo_r1/metrics/metric_api.py has two docstring drift bugs in
DistanceMetrics that mirror the pattern PR NVlabs#86 fixed in
distance_metrics.py / metric_utils.py:

1. DistanceMetrics.__init__ documents a `group_by_scenarios` parameter
   that does not exist in the signature (signature only has `prefix`
   and `time_step`). Replace the dead parameter entry with a real
   description of `time_step`, which has been undocumented since it was
   added.

2. DistanceMetrics.evaluate's Returns block lists `min_ade_sq` and
   `corner_distance_sq` (the "_sq" keys) which are never produced --
   the body only emits `min_ade`, `min_ade/by_t={H:.1f}`, `ade`,
   `ade/by_t=3.0`, `corner_distance`, plus `_std` variants from
   summarize_metric when `num_traj_sets > 1`. The current docstring
   also omits all of `ade*` and `*by_t*` keys entirely.

Pure docstring change. No code logic touched.

Verification: the new keys exactly match the body's `metric_dict.update`
calls (lines ~232, ~241, and the return of compute_grouped_corner_distance).

Signed-off-by: lonexreb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant