Skip to content

fix: avoid summing cumulative streaming usage on OpenAILike models - #11549

Closed
princejain756 wants to merge 1 commit into
agno-agi:mainfrom
princejain756:fix/openailike-streaming-metrics
Closed

princejain756 wants to merge 1 commit into
agno-agi:mainfrom
princejain756:fix/openailike-streaming-metrics

Conversation

@princejain756

Copy link
Copy Markdown
Contributor

Summary

Fixes #11538

OpenAI-compatible servers like vLLM (with force-include-usage or continuous usage stats) emit running cumulative token counts on intermediate SSE chunks and a final usage summary chunk with an empty choices list. Because OpenAILike inherited collect_metrics_on_completion = False, _populate_stream_data added every chunk's cumulative usage together, and enabling collect_metrics_on_completion still dropped the final choices: [] usage frame because _should_collect_metrics returned False when response.choices was empty.

This sets collect_metrics_on_completion: bool = True by default on OpenAILike, updates OpenAIChat._should_collect_metrics to accept the final empty-choices usage chunk, and replaces (while preserving timer, duration, and TTFT) rather than adds stream_data.response_metrics when collect_metrics_on_completion is enabled so streams that attach usage to both the finish_reason chunk and the trailing choices: [] chunk do not double-count.

Type of change

  • Bug fix

Checklist

  • Code complies with style guidelines
  • Ran format/validation scripts (./scripts/format.sh and ./scripts/validate.sh)
  • Self-review completed
  • Documentation updated (comments, docstrings)
  • Tested in clean environment
  • Tests added/updated (if applicable)

Duplicate and AI-Generated PR Check

  • I have searched existing open pull requests and confirmed that no other PR already addresses this issue

Additional Notes

Verified with pytest libs/agno/tests/unit/models/openai/test_openai_metrics.py libs/agno/tests/unit/models/perplexity/test_perplexity_metrics.py (23 passed).

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

PR Triage

Issue assigned to someone else: This PR references an issue that is already assigned:

Please confirm with the assignee or a maintainer in the issue comments before proceeding.


This PR has been automatically closed because the linked issue is already assigned to someone else. Only a maintainer can reopen it. Once you have addressed the points above (for example, a maintainer has marked the linked issue ready), leave a comment on this PR and a maintainer will review it for reopening. Please do not delete or force-push the branch in the meantime, otherwise the PR cannot be reopened.

@github-actions github-actions Bot closed this Nov 9, 2026
@princejain756

Copy link
Copy Markdown
Contributor Author

Hey @harshsinha03, sorry about that, I saw the ready label on #11538 and missed that you were already assigned. Leaving the branch up in case you want to reopen and use this fix (OpenAILike.collect_metrics_on_completion = True, accepting the final choices: [] usage chunk in _should_collect_metrics, and replacing stream_data.response_metrics when collect_metrics_on_completion is set).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant