-
Notifications
You must be signed in to change notification settings - Fork 3.4k
feat: show live token usage during streaming #4709
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
Enable token visibility to help users understand API costs and context window utilization. Display estimated tokens for user messages and actual output tokens plus cumulative context percentage for assistant messages. Include context panel toggle.
Display estimated output tokens (~X tok) during assistant message streaming using character-based estimation (4 chars = 1 token). Replace estimate with actual count on completion, with change detection to minimise message broadcasts.
Add real-time token estimation during streaming with separate tracking for output (~tok) and reasoning (~think) tokens. Accumulates estimates across multiple text/reasoning blocks within a step and across multiple steps (tool calls). Includes change detection to minimise message broadcasts and clean transition from estimates to actual counts on completion.
- Add sentEstimate/contextEstimate fields to message schema for tracking tokens sent to API - Extract tool result token calculation to shared helper - Replace magic number 4 with CHARS_PER_TOKEN semantic constant - Fix accumulation logic to always include user message tokens (not just as fallback) - Ensure tool result tokens are counted when sent back to API - Update TUI display to show accurate IN↓/OUT↑ token counts
Fix token counting logic to properly account for synthetic and noReply messages in session totals: - Account for synthetic content (e.g. skill documentation) in token counts by checking ignored flag instead of synthetic flag - Fix sentEstimate formula to prevent double-counting tool result tokens - Maintain cumulative session totals for accurate usage tracking - noReply messages now included in session token calculations This ensures users see accurate cumulative token usage per session without exponential growth from double-counting.
|
I like this 👍 |
|
looking forward to it! |
f8ee907 to
6a9856d
Compare
|
@rekram1-node any plans to include this feature in upcoming releases? |
…t#4709) Cherry-pick fix from upstream PR sst#4709 by arsham: - Use 'ignored' flag instead of 'synthetic' for accurate token counts - Fix sentEstimate formula to prevent double-counting tool result tokens - Add null checks in calculateToolResultTokens for safety - Prevents exponential growth from double-counting in session totals
* feat(tui): implement search functionality in session view - Added a search input component to allow users to search through messages. - Implemented logic to highlight search matches and navigate between them. - Updated the theme syntax rules to include strikethrough styling. - Enhanced message rendering to support search highlighting for text and markdown parts. * feat(tui): add bash output viewer with ANSI color support Adds an bash output viewer with full ANSI color support. When bash commands produce more than 20 lines of output, users can click to expand into a full-screen viewer with syntax highlighting and keyboard navigation. Key features: - Full ANSI/VT color rendering using opentui-ansi-vt terminal emulation - Truncated preview (20 lines) with 'Click to view full output' expansion - Prompt text preservation when entering/exiting viewer - Force color output in bash commands (FORCE_COLOR, CLICOLOR, TERM env vars) Technical changes: - Added opentui-ansi-vt dependency for terminal buffer rendering - Added initialValue prop to Prompt component to restore draft text - Strip ANSI codes for search to match actual text content * limit to MAX_OUTPUT_LENGTH * remove ANSI from model output with ptyToText * use 120 rows for terminal output * chore: update opentui-ansi-vt to 1.2.7 * feat(tui): add page up/down support to bash output viewer * fix(tui): improve match navigation and search highlighting - Refactored match navigation logic to prevent scrolling to the same message when navigating through matches. - Enhanced search highlighting to correctly identify and highlight query text within code blocks. - Updated rendering of Assistant messages to include IDs for better scrolling functionality during searches. * chore(tui-search): Removed active match differenciation, since we cant get it working in Assistant Messages (markdown) * feat: implement double Ctrl+C to exit functionality with warning toast * fix: improve token counting for synthetic and noReply messages (PR sst#4709) Cherry-pick fix from upstream PR sst#4709 by arsham: - Use 'ignored' flag instead of 'synthetic' for accurate token counts - Fix sentEstimate formula to prevent double-counting tool result tokens - Add null checks in calculateToolResultTokens for safety - Prevents exponential growth from double-counting in session totals * docs: update README for shuvcode fork - Add header explaining this is a fork for PR integration testing - Include table of merged PRs with upstream status - Preserve original opencode documentation below * changing sidebar version display to shuvcode
|
@arsham - found a bug in this one while using it in my fork: the upstream code has: assistantMessage.finish = "tool-calls" But your commit 7faa5e2 ("feat: real-time token tracking in TUI") accidentally removed completion handling code when refactoring the subtask execution. |
|
The implication of the bug is that custom /commands that have subtask: true set will complete it's task and then spawn another task trying to do the same thing, over and over until manual intervention |
This merge brings upstream changes while preserving fork-specific features: Preserved fork features: - PR sst#4898: Search in messages (Ctrl+F) - PR sst#4791: Bash output with ANSI colors - PR sst#4900: Double Ctrl+C to exit - PR sst#4709: Live token usage during streaming - PR sst#4865: Subagents sidebar with clickable navigation - PR sst#4515: Show plugins in /status - Fork-only: Toggle session header visibility Resolved conflicts: - Package versions updated to upstream v1.0.149 - Keybind type changes (cmd -> super) - Desktop context files updated with defensive API checks - UI components updated for new items prop signature - input.css/tsx renamed to text-field.css/tsx Validation: - All typechecks pass (12/12 packages) - All tests pass (281 passed, 1 skipped) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Resolved conflicts: - bun.lock: Regenerated from scratch - packages/*/package.json: Accept upstream version (1.0.167) - packages/extensions/zed/extension.toml: Accept upstream version Upstream changes include: - Version bump to 1.0.167 - Terminal title toggle feature added to config - MCP authentication improvements - Legal pages (privacy policy, terms of service) - Tauri package updates Fork features preserved: - PR sst#4898: Search in messages (Ctrl+F) - PR sst#4791: ANSI bash output support - PR sst#4900: Double Ctrl+C to exit - PR sst#4709: Live token usage during streaming - PR sst#4865: Subagents sidebar navigation - PR sst#4515: Show plugins in /status - PR sst#5508: Cache management commands - PR #ariane-emory: Granular file permissions - PR #fork-140: Toggle transparent background - Desktop features: slash commands, shell mode, themes, branding
Resolved conflicts: - script/publish.ts: kept fork version (fork-specific publishing to Latitudes-Dev/shuvcode) Auto-merged files with upstream improvements: - packages/opencode/src/cli/cmd/tui/routes/session/index.tsx: new dialog-subagent component - packages/opencode/src/session/message-v2.ts: improved error handling - packages/desktop/src/context/global-sync.tsx: added null directory guards Preserved fork features: - PR sst#4898: Search in messages (Ctrl+F) - PR sst#4791: Bash output with ANSI - PR sst#4900: Double Ctrl+C to exit - PR sst#4709: Live token usage during streaming - PR sst#4865: Subagents sidebar with clickable navigation - PR sst#5508: Cache management command - Glob permissions feature - Toggle transparent background - Desktop slash commands and share functionality
Resolved conflicts: - packages/opencode/src/cli/cmd/tui/routes/session/index.tsx: Merged fork search highlighting with upstream user message markdown toggle - packages/opencode/package.json: Accept upstream v1.0.170, preserved ghostty-opentui dependency - .github/workflows/generate.yml: Keep fork-specific workflow (better error messaging) - script/publish.ts: Keep fork-specific publish script (shuvcode registry, contributors) - packages/extensions/zed/extension.toml: Accept upstream v1.0.170 - All other package.json files: Accept upstream v1.0.170 versions - bun.lock: Regenerated Preserved fork features: - PR sst#4898: Search in messages (Ctrl+F with highlighting) - PR sst#4791: Bash output with ANSI (ghostty-opentui) - PR sst#4900: Double Ctrl+C to exit - PR sst#4709: Live token usage during streaming - PR sst#4865: Subagents sidebar with clickable navigation - PR #fork-140: Toggle transparent background - Fork: Toggle session header visibility - Fork: Desktop slash commands and shell input mode - Fork: Desktop Night Owl theme - Fork: Desktop theme/font pickers and branding - Fork: Share functionality for shuv.ai - Fork: Desktop mobile responsiveness - Fork: Desktop review pane resizing New upstream feature integrated: - User message markdown toggle (enable/disable markdown rendering in user messages)
Token counts only showed up after messages finished, so you had no idea how many tokens were being used until it was done.
Now token usage shows live while streaming:
IN↓/OUT↑format shows what you're sending vs receiving~X tokestimates appear immediately and update in real-time~X thinkshows reasoning token usage separatelyTool results are now properly counted when sent back to the API
Added
sentEstimateandcontextEstimatefields to track what's actually being sentMakes it way easier to keep an eye on token budget instead of being surprised at the end.
Showcase