You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: null is not an explicit reason — unavailable metrics must state why #238
Issue #215 acceptance criterion: "Captures token/sec where local runtime exposes it, or records an explicit unavailable reason."
PR #237 outputs "tokens_per_sec_by_model": {} and "total_completion_tokens": null when the backend does not return usage. This is not an explicit reason — it is an implicit absence. A reader of the smoke output cannot distinguish between:
Backend returned usage but values were 0
Backend did not return usage at all
Instrumentation hook was broken and never fired
This metric was not implemented
Root Cause
"null means unavailable" is not explicit: The criterion asks for an explicit reason, not implicit absence. An explicit reason would be: "token_throughput_status": "backend_omits_usage" or "token_throughput_unavailable_reason": "usage_field_absent".
Did not read the criterion literally: "or records an explicit unavailable reason" means the output must contain a field whose value explains why the metric is unavailable. null does not explain.
Prevention
For the agent
When a criterion says "explicit X", do not substitute implicit absence. An explicit field must exist in the output with a value that explains the situation.
Apply this pattern to every metric that can be unavailable: RAG latency ("not_invoked_no_embedding_client"), embedding latency ("not_invoked_no_embedding_client"), psutil RSS ("psutil_not_installed").
Activity