Skip to content

chore(desktop): add a packaged startup benchmark - #49758

Merged
Hona merged 8 commits into
v2from
desktop-startup-bench
Sep 18, 2026
Merged

Hona merged 8 commits into
v2from
desktop-startup-bench

Conversation

@Hona

@Hona Hona commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

The devex benchmark measures bun dev:desktop to Home. Nothing measured what users launch: a packaged build restoring its tabs against an already-running service. This adds bun run bench:startup.

flowchart LR
  A["packaged exe<br/>(dist/win-unpacked)"] --> B["isolated home<br/>%APPDATA%, XDG, DB, config, service registration<br/>OPENCODE_* / OTEL_* stripped"]
  B --> C["warm service (bundled CLI, once)<br/>or cold: spawned per launch"]
  C --> D["N launches, interleaved A/B with --compare<br/>1 warm-up launch discarded"]
  D --> E["milestones since spawn:<br/>process created · Node start · Node bootstrapped · app starting<br/>service ready · renderer process · window visible · first paint · shell · idle"]
  D -. --profile-main .-> F["main CPU profile from first statement"]
  D -. --profile-renderer .-> G["renderer main-thread CPU profile"]
  D -. --trace .-> H["Chromium --trace-startup"]
Loading

What it isolates. Own %APPDATA%/XDG/DB/config/service registration (the desktop deletes XDG_STATE_HOME on Windows, so isolation goes through the home directory), the developer's OPENCODE_*/OTEL_* environment stripped (an inherited OTLP endpoint added a Sentry round trip to every CLI exit inside the measured app), a private service port for cold mode, and it only ever kills the process tree it spawned (a prod-channel build shares its executable name with the installed app).

What it measures. Main-log milestones, renderer paint timing and DOM readiness over CDP, renderer idle (first 500 ms window under 10 % main-thread task time that stays quiet for --settle-ms) and total renderer task time. After each run it reads process.getCreationTime() and performance.nodeTiming from the main process over --inspect (nothing attaches until the run is over), which splits the time before our first log line into Electron native init, Node bootstrap, and Electron JS init + our bundle — so a fat main bundle can't hide inside a "Chromium boot" number.

Per-run detail (in every sample of the JSON report): the merged main-process log timeline — main plus the scoped crash/onboarding/window/renderer logs as [ms, file, message] — the renderer's main-thread breakdown (script, layout, style, counts, DOM nodes, JS heap), navigation/resource/paint timing and any performance.marks, the working set of every process in the app's tree once idle, and the main process's CPU time. The summary adds median phase durations between consecutive checkpoints:

electron native init                55
node bootstrap                      18
electron js init + main bundle     412
main layers → window created       131
renderer boot → first paint        524
first paint → shell                113
shell → idle                        68

memory once idle — total 518 MB · main cpu 1468 ms · renderer heap 17 MB, 1318 nodes, layout 41 ms/11×, style 53 ms/87×
main 154 · gpu-process 106 · utility 54 · crashpad-handler 46 · renderer 141 · ssh.exe 10 · conhost.exe 7
```n
**Modes.** `--service warm|cold`, `--fresh` (profile wiped per launch: first launch after install), `--compare <exe>` (alternating launches so drift hits both builds equally), `--warmup N` (discarded launches; the first launch of a new binary pays the antivirus scan), `--offline` (implicit for beta/prod), `--seed <userData>` (restore real tabs/drafts), `--profile-main`, `--profile-renderer`, `--trace`.

Sample, packaged `v2` build vs the startup PR set, same machine, interleaved, warm service, median of 4 (ms since spawn):

| milestone | v2 | startup PRs |
|---|---|---|
| process created | 9 | 9 |
| Node starts (Electron native init done) | 49 | 51 |
| Node bootstrap complete | 63 | 63 |
| first main-process log line | 610 | 394 |
| service ready | 875 | 588 |
| renderer process | 920 | 511 |
| window visible | 1,273 | 786 |
| first paint | 1,440 | 959 |
| shell visible | 1,496 | 1,067 |
| renderer idle | 1,550 | 1,127 |
| renderer main-thread task time | 595 | 549 |

(This run was taken on a loaded machine; the interleaving is what makes the comparison valid.)

bench:startup launches a packaged build in an isolated home (own appData,
XDG directories, database, config and service registration) against a
warm or cold bundled service, and records milestones from process spawn:
main log events, renderer paint timing, shell / composer / timeline
readiness polled over CDP, and renderer idle. Optional main-process CPU
profile from the first statement and a Chromium startup trace. The devex
benchmark covers the dev server; this covers what users launch.
Copilot AI lite review requested due to automatic review settings September 18, 2026 12:28
@Hona
Hona requested a review from Brendonovich as a code owner September 18, 2026 12:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

longtask entries are not buffered, so idle was detected against the
probe time. Poll Performance.getMetrics TaskDuration alongside DOM
readiness instead: idle is the start of the first 500 ms window with
under 10 % task time that stays quiet for --settle-ms, and the total
task and script time are reported as the renderer's CPU cost. Also keep
the app alive until Chromium has written the startup trace.
…ained

- --service cold gives the spawned service a private port through the
  isolated config directory instead of requiring no other service
- kill only the process tree the run spawned; a prod-channel build shares
  its executable name with an installed app
- optional renderer CPU profile (--profile-renderer) from the moment the
  page target appears
- wait for Chromium to write the startup trace before quitting
…p bench

- read process creation and Node bootstrap timing from the main process
  after each run over --inspect, so the time before the first log line
  splits into Electron native init, Node bootstrap, and Electron JS init
  plus our bundle
- strip the developer's OPENCODE_* / OTEL_* environment: an inherited
  OTLP endpoint added a Sentry round trip to every CLI exit inside the
  measured app
- --compare alternates two builds so machine drift hits both equally;
  --warmup discards the first launch of each binary (antivirus scan)
- --fresh wipes the profile per launch for the first-launch experience
- --offline (implicit for beta/prod) keeps the updater's first check off
  the network
Each sample now carries the merged main-process log timeline (main plus
the scoped crash/onboarding/window/renderer logs, as ms since spawn),
the renderer's main-thread breakdown from Performance.getMetrics
(script, layout, style, counts, DOM nodes, JS heap), navigation, resource
and paint timing plus any performance marks, the working set of every
process in the app's tree once idle, and the main process's CPU time.
The summary adds median phase durations between consecutive checkpoints.
@Hona
Hona merged commit 98c6b41 into v2 Sep 18, 2026
10 checks passed
@Hona
Hona deleted the desktop-startup-bench branch September 18, 2026 23:23
jinhuang712 pushed a commit to jinhuang712/opencode that referenced this pull request Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants