Repository navigation
Commit dad1b5d
build(perf): migrate benchmark harness to Babashka (#1432)
* build(perf): migrate benchmark harness to Babashka
Replace the large Nushell PR benchmark script with a Babashka implementation split by data, system, benchmark, report, and orchestration responsibilities. The new process boundary keeps argv, environment, and working-directory data explicit while preserving hyperfine, package installation, memory, size, and Markdown behavior.
Move the CI caller and profiling guidance to the executable Babashka entry point. Add focused tests behind their own Nix shebang so contributors can run the harness suite without adding Babashka to the full development shell.
* docs(agents): document implementation language choices
Route small command-oriented automation to Nushell and data-heavy, testable automation to Babashka. Keep production binaries in Rust and npm-integrated APIs in TypeScript so future tooling changes follow the same criteria used by the benchmark migration.
* test(ci): run Babashka harness tests
Execute the self-contained benchmark harness test entry point in the CI test job so changes to CLI parsing, normalization, fallback decisions, and report rendering cannot bypass pull request validation.
* fix(perf): harden platform and tarball paths
Normalize version-qualified Windows os.name values to win32 so native executable and package paths use the expected suffixes. Resolve relative pnpm pack filenames against the temporary destination while preserving the absolute paths emitted by current pnpm versions.
Add regression coverage for both platform normalization and relative or absolute tarball filenames.
* fix(perf): size local package fallbacks
Use remote tarball sizing only after the corresponding preview package was installed successfully. When either package URL times out, benchmark and size the available local checkout so fallback runs can still produce a complete report.
Cover base and head source selection and verify both unavailable URLs through a committed-fixture smoke run.
* fix(perf): bound harness child processes and skip RSS on unsupported platforms
Add a cancellable timeout to run-process and thread --package-runner-timeout-ms through the package URL probe, install, pnpm pack, and git rev-parse flows so a stalled child cannot outlive the deadline; give the curl probe and download explicit connect and read limits.
measure-memory now warns once and skips gracefully when /usr/bin/time is unavailable (unsupported platforms) instead of throwing and aborting the entire benchmark run.
Co-authored-by: Codesmith <[email protected]>
* Revert "fix(perf): bound harness child processes and skip RSS on unsupported platforms"
This reverts commit 9140a99.
---------
Co-authored-by: Codesmith <[email protected]>1 parent 997ad7f commit dad1b5d
13 files changed
Lines changed: 696 additions & 1057 deletions
File tree
- .agents/skills/profile/references
- .github/workflows
- apps/ccusage
- scripts
- compare_pr_performance
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
| 75 | + | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | | - | |
| 81 | + | |
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
179 | 179 | | |
180 | 180 | | |
181 | 181 | | |
| 182 | + | |
| 183 | + | |
182 | 184 | | |
183 | 185 | | |
184 | 186 | | |
| |||
492 | 494 | | |
493 | 495 | | |
494 | 496 | | |
495 | | - | |
| 497 | + | |
496 | 498 | | |
497 | 499 | | |
498 | 500 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
36 | 40 | | |
37 | 41 | | |
38 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
35 | 39 | | |
36 | 40 | | |
37 | 41 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
0 commit comments