Repository navigation
Description
Affects: opencode serve PersistentPty + opencode-pty daemon 0.3.1, protocol 7.
Summary
With ~25 persistent PTYs on one host, each keystroke takes ~160 ms inside opencode serve before the
daemon writes it, and the idle opencode-pty daemon uses ~28 % of a core, and every request waits up to 10 ms to be accepted. Three causes:
-
serve calls
list()before every op.PersistentPty.input(and write, resize, control, snapshot)
lists every terminal to resolve one id. The daemon fills each terminal'sforeground_processby
sweeping/proc, ~7 ms a terminal, serially. A key costs O(terminals) of/procsweeps. -
The daemon sweeps
/procper terminal on its own. Even with no requests, each terminal's thread
walks all of/procabout every 10 s (28 sweeps in 10 s on a 28-PTY host, ~10 kreads a second). -
The daemon polls
accept()every 10 ms. Every one-shot request (serve's included) waits 0–10 ms
before the daemon reads it: on the live daemon,accept4runs ~100 times a second alongside
clock_nanosleep(~90/s) andsched_yield(~700/s) while idle. Requests phase-lock to a 10 ms grid
(pingandinputsent 37 ms apart all take 3.3 ms; spread out, p50 0.75 / p90 9.9 ms). A
connection opened ≥10 ms before its request is written answers in 0.2 ms.
Measurements (Linux 7.0 Azure VM, 8 vCPU, 24–28 PTYs, eBPF + strace)
| value | |
|---|---|
| key first byte, client ws → serve → daemon | 167 ms p50 |
| key first byte, client → daemon socket directly | 7 ms p50 |
daemon input op itself |
0.1 ms |
| idle daemon CPU (60 s, no typing) | 28.5 % (22.7 % sys) |
serve list requests reaching the daemon, idle |
1 every 5 s |
ping/input on a fresh connection (1 PTY) |
p50 0.75 ms, p90 9.9 ms (10 ms accept poll) |
| same request on a connection opened 50 ms earlier | 0.2 ms |
| key → echo, opencode TUI in a PTY, bare (sandbox, 364x91) | p50 9–11 ms; the daemon's accept poll adds p50 5.6 / p90 9.4 ms |
Adding 16 idle PTYs moved typing latency 136 → 18 ms (shell) and 141 → 30 ms (opencode TUI) once input
bypassed serve, so the cost scales with the terminal count, not the typing.
Suggested fix
- serve: resolve a terminal by id without
list()on input/write/resize/control/snapshot (keep a map
from the attach, or call a per-id op on the daemon). - daemon: block in
accept()(or poll the listener with the connections) instead of a 10 ms sleep loop,
and serve several requests per connection so a client can keep one open. - daemon: find the foreground process with
tcgetpgrp(master_fd)and read/proc/<pgid>/{stat,cmdline},
instead of sweeping all of/proc; compute it lazily (only whenlistasks), not on a per-terminal timer.
Workaround in use
olympus sends keystrokes straight to the daemon's unix socket (protocol 7 input op), falling back to
serve's websocket. It depends on a private protocol and will be retired once this is fixed. It also keeps one spare
connection open ahead of the next key to dodge the accept poll.
Activity
opencodejr commentedon Oct 30, 2026
Thanks for the detailed measurements. I could reproduce the main part of this.
The script below creates persistent terminals through
opencode serveand asks for the snapshot of one terminal. It counts the daemon's/procopens for that single request:/procopens (2.2.4, daemon 0.4.0)This still happens on the latest
dev(171dbd6).PersistentPty.get(id)still sends the daemon'slistop for each per-terminal operation, andliststill sweeps all of/proconce per terminal to find its foreground process.The other two parts did not show up on the current daemon (0.4.0). With 50+ terminals and no requests, the idle daemon made no
openat,accept4orclock_nanosleepcalls in 10 s. The polling was removed in #59701 and #59698, after the 0.3.1 release you measured. Upgrading should take care of the idle CPU and the 0–10 ms accept delay; the cost of each op still grows with the number of terminals.Reproduction script (repro.sh)
Run it:
bash repro.sh