You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f552d87
Browse filesBrowse the repository at this point in the historyBrowse files
feat(perf): time every Android command against the fixture app (#3902)
* feat(test-app): add a Bluesky-depth heavy feed screen
A JavaScript-only route that reproduces a heavy React Native home feed: 30
nested containers above a virtualized list of composite posts with image,
link, quote, and quote-with-media embeds. Its native accessibility tree is
deeper than 65 levels, so XCTest tree capture fails with
kAXErrorIllegalArgument and recovers through private AX, as on Bluesky
1.136. Settings links to it with open-heavy-feed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* feat(perf): time every iOS command against the fixture app
pnpm perf --platform ios now drives the test app through seven scenarios that
time each catalog command the iOS simulator serves, including the heavy feed
and a raw capture with the XCTest tree backend pinned. The suite records a
reason for every command it does not time, and a unit test keeps that
partition complete.
Each sample keeps the client daemon round trip and a phase profile from its
--debug request log: runner commands, host process spawns, snapshot sources,
retries, stabilization, and fallbacks. A run writes summary.json, report.md,
and the raw daemon and session logs. The iOS Settings tour is removed; the
Android tour stays as it was.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* feat(perf): compare nightly iOS runs and run the suite every night (#3886)
pnpm perf:compare judges one run against earlier runs: by default the seven
nights before it, or a chosen run, date, range, or local output. A step's
baseline is the median of its nightly medians. A change is flagged only when
it is larger than 20%, three times the night-to-night scaled MAD, twice the
run's own cv/sqrt(n), and 50 ms; new failures and fixed steps are flagged
too.
perf-nightly runs the iOS suite at 02:40 UTC again (#1781 A3). Each run uploads
perf-ios-summary (90 days) and perf-ios-raw (30 days), and writes the
comparison to the job summary with warnings for regressions and new
failures. Android stays report-only behind a dispatch input.
Co-authored-by: Claude Opus 5.5 <[email protected]>
* fix(perf): judge nights, samples, and failures soundly; keep logs on interrupt
- Rolling baselines take one scheduled run per UTC night (the newest
usable one) and page through workflow runs until enough nights are
found. Explicit --to/--from references still accept any trigger.
- A run measures at least three rounds. A timing verdict needs three
successful samples tonight and on each counted baseline night.
- Failures get their own verdict. more-failures flags tonight failing
more than every baseline night, in count and share; a faster median
with worse failures is not an improvement.
- The interrupt handler and the normal finish share one finalizer:
save logs, stop session and daemon within a deadline, save again.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* feat(perf): time every Android command against the fixture app
Run iOS and Android from one shared suite: platform deltas are data on
each step, and one harness owns rounds, sessions, and daemon restarts.
The heavy-feed scenario times the Android helper's one-shot
instrumentation capture by restarting the daemon with the helper
session off. Request profiles name the helper transport that answered,
adb spawns, and helper gestures. The nightly runs both platforms and
compares each against its own nights; the Settings tour is removed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): keep Android clipboard and restore failures out of timings
The API 35 and 36 emulator images ship no shell clipboard service, so
clipboard is an Android exclusion. A daemon restart that reopens the
session clears the stuck flag, and one that cannot reopen it skips the
rest of the scenario as restore_failed. The one-shot step warms the
restarted daemon's helper install check untimed. latest and date
references resolve to the newest run with the platform's artifact.
The tour-only --n alias is gone.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* feat(perf): time every iOS command against the fixture app
pnpm perf --platform ios now drives the test app through seven scenarios that
time each catalog command the iOS simulator serves, including the heavy feed
and a raw capture with the XCTest tree backend pinned. The suite records a
reason for every command it does not time, and a unit test keeps that
partition complete.
Each sample keeps the client daemon round trip and a phase profile from its
--debug request log: runner commands, host process spawns, snapshot sources,
retries, stabilization, and fallbacks. A run writes summary.json, report.md,
and the raw daemon and session logs. The iOS Settings tour is removed; the
Android tour stays as it was.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* feat(perf): compare nightly iOS runs and run the suite every night (#3886)
pnpm perf:compare judges one run against earlier runs: by default the seven
nights before it, or a chosen run, date, range, or local output. A step's
baseline is the median of its nightly medians. A change is flagged only when
it is larger than 20%, three times the night-to-night scaled MAD, twice the
run's own cv/sqrt(n), and 50 ms; new failures and fixed steps are flagged
too.
perf-nightly runs the iOS suite at 02:40 UTC again (#1781 A3). Each run uploads
perf-ios-summary (90 days) and perf-ios-raw (30 days), and writes the
comparison to the job summary with warnings for regressions and new
failures. Android stays report-only behind a dispatch input.
Co-authored-by: Claude Opus 5.5 <[email protected]>
* fix(perf): judge nights, samples, and failures soundly; keep logs on interrupt
- Rolling baselines take one scheduled run per UTC night (the newest
usable one) and page through workflow runs until enough nights are
found. Explicit --to/--from references still accept any trigger.
- A run measures at least three rounds. A timing verdict needs three
successful samples tonight and on each counted baseline night.
- Failures get their own verdict. more-failures flags tonight failing
more than every baseline night, in count and share; a faster median
with worse failures is not an improvement.
- The interrupt handler and the normal finish share one finalizer:
save logs, stop session and daemon within a deadline, save again.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): pin the tree backend on a capture it serves, and share host readers
- The forced-tree step ran a raw capture. The runner honors a backend pin only on the
regular plan, so the raw plan recovered through private AX and the expectation passed
a degraded row. The step is now a regular pinned capture on the home screen, and its
expectation requires the pinned backend with the runner's `requested-backend` code.
- Perf metadata reads the revision, toolchain, and simulator through the readers of
the iOS snapshot benchmark, moved to scripts/lib/bench-host.ts.
- host-kit and the daemon client export the phase names that the profiler reads.
- summary-fixture.ts becomes summary.fixtures.ts; the nightly header matches 3 rounds.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): read Android phase names from their emitters, and document the pinned capture
- platform-android exports the helper capture and gesture phase names through
its mechanics entry; the profiler and its test import them.
- The README's snapshot-path section and parity table name the iOS pinned
capture on home-reads and the reason code it must carry.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): name the failed restore when restarting the daemon, and fix the local compare example
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): keep the daemon request phase literal
Only scripts read it, so a root export has no production consumer and fails the
fallow audit.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(perf): stop the log stream right after it starts
#3892 fixed the iOS simulator logs stop hang, so the untimed wait before
logs-stop is no longer needed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* refactor(android): record the answering helper transport on its capture timer
The snapshot helper's capture timers now name the transport they drove, and the
session timer records whether the session answered. profile.ts reads those
fields per event instead of inferring a silent session from event order, and no
longer re-spells the producer names platform-android owns.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
0 commit comments