feat(agent): per-call output speed (tokens/sec) for desktop status bar and runtime footer by beresfordsalmon · Pull Request #44878 · NousResearch/hermes-agent

beresfordsalmon · 2026-06-12T11:50:47Z

Summary

Stores per-API-call timing and token counts on the agent so downstream consumers (desktop status bar, gateway runtime footer, CLI status bar) can display real-time tokens-per-second.

run_agent.py: Initialize last_api_duration / last_output_tokens on agent
agent/conversation_loop.py: Persist per-call api_duration + output_tokens after each successful API response
tui_gateway/server.py: Compute output_speed in _get_usage() (read by desktop session.info events and session.usage RPC)
cli.py: Expose output_speed in TUI status bar when display.show_output_speed is enabled
gateway/runtime_footer.py: Add tps field for gateway platforms (messaging, desktop response footer)
gateway/run.py: Wire per-call stats into agent result + footer call site
hermes_cli/config.py: Add display.show_output_speed toggle (default off)

Validation

Verified end-to-end: conversation_loop.py SETs last_api_duration and last_output_tokens (confirmed via debug log)
tui_gateway/server.py _get_usage() reads them and emits output_speed (confirmed 48.0 tok/s for DeepSeek v4 Pro)
Desktop frontend already has output-speed status bar item (needs rebuild to display)

Closes #9527
Closes #26877

…r and runtime footer Store last_api_duration and last_output_tokens on the agent after each API call so the gateway can compute output_speed for the desktop status bar and runtime footer tps field. - run_agent.py: initialize last_api_duration / last_output_tokens - agent/conversation_loop.py: persist per-call timing + token counts - tui_gateway/server.py: compute output_speed in _get_usage() - cli.py: expose output_speed in CLI status bar (show_output_speed config) - gateway/runtime_footer.py: add tps field for gateway platforms - gateway/run.py: wire per-call stats into footer + agent result - hermes_cli/config.py: add display.show_output_speed toggle (default off) Closes #9527

…PI response

- Add /tps to command registry - Add _show_tps handler in CLI - Add session.tps RPC in gateway server - Pass gateway port + session_id to slash worker - Slash worker calls session.tps when local agent has no data

…eate was missing it)

…n agent

…handles /tps directly)

tonydwb

Good feature addition. Adds per-call output speed (tokens/sec) display to desktop status bar, runtime footer, and /tps CLI command. Tracks last_api_duration and last_output_tokens per call. Clean implementation across 11 files. No issues found.

peak_speed = (last_output_tokens + session_reasoning_tokens) / last_api_duration This is the true decode rate including hidden reasoning tokens. - _get_usage() now emits peak_speed in every session.info event - CLI _get_status_bar_snapshot() also exposes peak_speed - peak_speed is available to the desktop status bar via currentUsage?.peak_speed (frontend must opt-in: s/output_speed/peak_speed in use-statusbar-items.tsx)

fleps · 2026-08-12T14:55:04Z

I was thinking on doing a plugin for my Desktop then found this PR.
This should really be an OOTB feature.

spfcraze · 2026-08-13T13:18:47Z

This was generated by AI during triage.

Summary:
The peak_speed figures (in _get_usage and the /tps "Peak speed (est)" line) add the session-cumulative session_reasoning_tokens to the last call's output but divide by only that call's duration, so the rate inflates with every prior reasoning-heavy call.

Problems:
session_reasoning_tokens is a running session total (agent.session_reasoning_tokens += canonical_usage.reasoning_tokens per call, reset to 0 at session start), while last_api_duration and last_output_tokens describe only the most recent call. In tui_gateway/server.py the _get_usage peak-speed branch (around line 2150) sets total_out = last_out + reasoning, where reasoning is that session total, then divides by last_dur. After a few reasoning-heavy calls the numerator keeps growing while the denominator stays a single call, so peak_speed grows without bound and can dwarf the actual rate. The /tps handler (around line 8761) and the CLI's status-bar snapshot and /tps command repeat the same last_out + session_reasoning_tokens over last_dur arithmetic, so every client surfaces the same inflated number.

Solution:
Record the last call's own reasoning tokens (e.g. set last_reasoning_tokens = canonical_usage.reasoning_tokens alongside last_output_tokens in the same per-call block) and compute the peak from (last_output_tokens + last_reasoning_tokens) / last_api_duration. If a session-averaged figure is intended instead, pair the cumulative reasoning numerator with the session-cumulative duration, not the single call's.

Evidence

no deterministic fact backs this claim — model belief, not executed or read evidence

Checked against b8e69c7 — the tip of feat/tps-display when this was written — and fa83af3, main at the same moment.

Resolve conflicts from two months of main drift: - gateway/runtime_footer.py: keep both turn_seconds (main) and response_tokens/elapsed_ms (PR) params on build_footer_line. - gateway/run.py: re-apply last_output_tokens/last_api_duration capture inside the extracted TurnRunner result path (run_sync moved to gateway/turn_runner ctx flow on main). - hermes_cli/config_defaults.py: DEFAULT_CONFIG moved out of config.py; add display.show_output_speed there. - tui_gateway: server.py split into methods_*.py on main — re-apply session.tps RPC in methods_session.py and the direct /tps handler in methods_tools.py slash.exec; keep _get_usage output_speed/peak_speed. Drop dead HERMES_SLASH_GATEWAY_PORT env plumbing (superseded by the direct slash.exec handler within the PR's own commit history). - cli.py: re-apply /tps dispatch + _show_tps + status-bar t/s segment on top of the battery-item refactor. - apps/desktop i18n: add missing shell.statusbar.tokensPerSecond key (en/zh/zh-hant/ja/ar + types).

…issing i18n - tui_gateway/server.py: numeric-coerce last_api_duration/last_output_tokens in _get_usage so partially-built agents and test doubles (MagicMock) can't crash the usage snapshot with a TypeError on comparison. - hermes_cli/commands.py: route /tps via /hermes tps on Slack (_SLACK_VIA_HERMES_ONLY) — a native slot at the 50-slash cap would clamp /insights and break Telegram parity. - contributors/emails: map mcsal@local -> beresfordsalmon (CI attribution gate).

teknium1 · 2026-08-18T00:20:19Z

Rebased this onto current main (merge commit, contributor commits preserved) and hardened it as part of the weekly Kilo Code scout run (their #12434/#13164 ship the same tok/s display, which independently validates the feature):

Resolved ~2 months of drift: DEFAULT_CONFIG moved to config_defaults.py, tui_gateway/server.py split into methods_*.py (re-applied session.tps RPC in methods_session.py and the direct /tps handler in methods_tools.py), gateway run_sync extraction into TurnRunner, battery status-bar refactor in cli.py.
Dropped the dead HERMES_SLASH_GATEWAY_PORT env plumbing (superseded within this PR's own history by the direct slash.exec handler).
Hardened _get_usage with numeric coercion (MagicMock/partial agents crashed two compression tests).
Routed /tps via /hermes tps on Slack — a native slot at the 50-slash cap would clamp /insights.
Added the missing shell.statusbar.tokensPerSecond i18n key (en/zh/zh-hant/ja/ar + types).

Targeted suites green: runtime_footer + tui_gateway_server + commands + turn_summary (726 passed). Awaiting Teknium's review.

abdulchotu7 · 2026-08-25T03:59:26Z

Please merge this ASAP

fleps · 2026-08-30T02:04:34Z

@beresfordsalmon why was this closed? 😞

McSal added 7 commits June 12, 2026 12:50

feat(cli): add /tps slash command to show tokens-per-second of last A…

4368ac0

…PI response

feat(cli): /tps slash command via gateway RPC for desktop app

936e513

- Add /tps to command registry - Add _show_tps handler in CLI - Add session.tps RPC in gateway server - Pass gateway port + session_id to slash worker - Slash worker calls session.tps when local agent has no data

fix(cli): verbose /tps error reporting for gateway RPC failures

861fbf2

fix(gateway): pass session_id to all _SlashWorker call sites (lazy-cr…

79184f1

…eate was missing it)

fix(gateway): handle /tps directly in slash.exec, reading from sessio…

7a47465

…n agent

refactor(cli): remove dead HTTP RPC fallback from _show_tps (gateway …

debb992

…handles /tps directly)

feat(tps): add reasoning token count to /tps output

7e5d10e

tonydwb approved these changes Jun 12, 2026

View reviewed changes

McSal added 2 commits June 12, 2026 13:42

feat(tps): add peak speed estimate (output + reasoning) / total time

714f718

alt-glitch mentioned this pull request Jun 25, 2026

feat(gateway): add token usage to runtime footer #52288

Open

teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 14, 2026

This was referenced Jul 28, 2026

[Feature]: Add tps (tokens per second) field to runtime_footer #26877

Open

[Feature]: Add display.show_output_speed to show tokens-per-second after each response #9527

Open

teknium1 added 2 commits August 17, 2026 17:17

alt-glitch added comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/desktop Electron desktop app (apps/desktop/*) area/config Config system, migrations, profiles area/usage-cost labels Aug 18, 2026

beresfordsalmon closed this by deleting the head repository Aug 29, 2026

phpoh mentioned this pull request Sep 8, 2026

[FEATURE]: Real-time token generation speed (tok/s) in TUI status bar, like Hermes anomalyco/opencode#47914

Open

alt-glitch added area/billing Account usage, credit usage, billing (cross-cutting) and removed area/usage-cost labels Oct 17, 2026

Navigation Menu

Repository navigation

feat(agent): per-call output speed (tokens/sec) for desktop status bar and runtime footer - #44878

feat(agent): per-call output speed (tokens/sec) for desktop status bar and runtime footer#44878
beresfordsalmon wants to merge 12 commits into
NousResearch:mainfrom
beresfordsalmon:feat/tps-display

beresfordsalmon commented Jun 12, 2026

Uh oh!

tonydwb left a comment

Uh oh!

fleps commented Aug 12, 2026

Uh oh!

spfcraze commented Aug 13, 2026

Uh oh!

teknium1 commented Aug 18, 2026

Uh oh!

abdulchotu7 commented Aug 25, 2026

Uh oh!

fleps commented Aug 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

beresfordsalmon commented Jun 12, 2026

Summary

Validation

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Uh oh!

fleps commented Aug 12, 2026

Uh oh!

spfcraze commented Aug 13, 2026

Evidence

Uh oh!

teknium1 commented Aug 18, 2026

Uh oh!

abdulchotu7 commented Aug 25, 2026

Uh oh!

fleps commented Aug 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants