Skip to content

Fix MetricRunner.run() return-type annotation and document side effect#92

Open
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:fix/metric-runner-return-type
Open

Fix MetricRunner.run() return-type annotation and document side effect#92
lonexreb wants to merge 1 commit intoNVlabs:mainfrom
lonexreb:fix/metric-runner-return-type

Conversation

@lonexreb
Copy link
Copy Markdown
Contributor

@lonexreb lonexreb commented May 4, 2026

Problem

MetricRunner.run in src/alpamayo_r1/metrics/metric_runner.py declares -> dict[str, Any] but the body has no return statement:

def run(
    self, model: Any, data_batch: dict[str, Any], output_batch: dict[str, Any]
) -> dict[str, Any]:
    """Run the metrics one-by-one by the order of the metrics list."""
    per_sample_metrics = {}
    for metric in self.metrics:
        per_sample = metric.evaluate(model, data_batch, output_batch)
        per_sample_metrics.update(per_sample)

    output_batch.update({"metric/" + k: v for k, v in per_sample_metrics.items()})

Python returns None implicitly while the annotation lies. This trips up type checkers (mypy, pyright) and is misleading to callers who reasonably assume they should read a result from the return value.

The actual contract — documented in the class docstring ("…added to the output batch") and matching every existing call site — is to merge the prefixed metrics into output_batch in-place.

Fix

  • Change the return annotation from dict[str, Any] to None so the type matches the runtime behavior.
  • Expand the one-line docstring to spell out the in-place mutation contract and explicitly tell callers to read results from output_batch after the call.

No runtime behavior change.

Verification

grep -A 12 "def run" src/alpamayo_r1/metrics/metric_runner.py shows the body has no return statement; the only side effect is the output_batch.update(...) call on the last line.

`MetricRunner.run` declares `-> dict[str, Any]` but the body has no
`return` statement -- Python returns `None` implicitly while the
annotation lies, which trips up type checkers and confuses callers
who assume they should read a result from the return value.

The body's actual contract is:

    output_batch.update({"metric/" + k: v for k, v in per_sample_metrics.items()})

i.e. metrics are merged into `output_batch` in-place. This is the
documented behavior in the class docstring ("...added to the output
batch") and matches every existing call site.

This commit:

- Changes the return annotation from `dict[str, Any]` to `None` to
  reflect the actual behavior.
- Expands the one-line docstring to spell out the in-place mutation
  contract and explicitly note that callers should read results from
  `output_batch` after the call.

No runtime behavior change.

Signed-off-by: lonexreb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant