Skip to content

fix(core): include provider wait in token throughput timing - #47125

Closed
opencode-agent[bot] wants to merge 1 commit into
v2from
token-rate-aiden
Closed

opencode-agent[bot] wants to merge 1 commit into
v2from
token-rate-aiden

Conversation

@opencode-agent

@opencode-agent opencode-agent Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix inflated TUI output-token throughput for responses that arrive in a burst after waiting on the provider.

The denominator is the sum of time.streamed - time.created across the assistant's model steps. Previously, time.created came from the lazily published step-start event, which can arrive only after the provider has already spent most of the request generating the response. A deterministic 12-token fixture with 2,690 ms of provider wait and 10 ms of response delivery therefore reports 1,200 tok/s instead of about 4.4 tok/s across the request.

  • Capture request start after snapshot preparation and before consuming the provider stream; carry it as an optional started timestamp on the existing step-start event.
  • Use that timestamp in both durable and live client message projections. Preserve lazy event publication so overflow/full-context recovery does not create extra assistant messages.
  • Reset the timestamp for a retried physical attempt so failed attempts and backoff do not enter the successful attempt's denominator.
  • Keep the response-body end boundary, excluding local tool settlement time. This is request throughput including provider wait, not decode-only speed.
  • Preserve older events without the new field. Missing historical request times cannot be reconstructed; this corrects newly recorded steps without inventing a backfill.

Validation

  • Reproduced before the fix: buffered-response test expected a 2,700 ms interval and received 10 ms. It now passes, including durable replay.
  • Core runner, publisher, message-update, and projector tests: 230 passed.
  • Live client data tests: 25 passed, including new/retried messages, zero timestamps, and legacy fallback.
  • TUI rows/data tests: 70 passed; rendered footer lifecycle test: 1 passed.
  • Schema event/manifest tests: 12 passed, including optional-field encoding compatibility.
  • Core, client, and schema typechecks passed; clients regenerated; formatting and diff checks passed.
  • The broader schema contract-hygiene check still fails on existing Schema.Any usage. Confirmed the same failure with the schema source byte-identical to the base commit; not changed in this patch.

Requested by: @rekram1-node (Aiden via Slack)

dukebw added a commit to dukebw/opencode that referenced this pull request Sep 17, 2026
# What/why

Include reasoning tokens in the tok/s indicator and measure their full request
window. Provider stream events can arrive after hidden reasoning, and retaining
an earlier attempt's creation timestamp includes retry backoff in the rate.

# How

Align request timing with anomalyco#47125: capture the request-start timestamp after
snapshot preparation, carry epoch milliseconds on Step.Started, and project it
into time.created in both durable and live client state. Reset it for each
physical retry and preserve the existing fallback for historical events without
a start timestamp.
Keep lazy assistant creation and the response-body end boundary before local
tool settlement. Sum visible output and reasoning over streamed - created.

# Testing

316 focused tests passed across core runner/projector/publisher, live client
projection, schema/protocol event, TUI row, and rendered footer regression suites.
Full bun run check passed. Regenerated clients and verified stable output.

stack-pr-branch: dukebw/stack/0681e582ab9d4b2ab73deedf3a7b43a8
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Automated PR Cleanup

Thank you for contributing to opencode.

Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions.

This PR was closed because it matched the following cleanup criteria:

  • The PR was created more than 1 month ago
  • The PR had fewer than 2 positive reactions
  • Positive reactions are counted as thumbs-up, heart, celebration, or rocket reactions on the PR

PRs created within the last month are not affected by this cleanup.

If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate.

Thanks again for taking the time to contribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant