Repository navigation
Commit 9372b11
benchmarks: a host without ps is not read as quiet; run_all.sh records the busy count as unknown and labels the figures quiet-unverified (#387)
benchmarks: a host without ps is not read as quiet; run_all.sh records the busy count as unknown and labels the figures quiet-unverified
The benchmark test file runs the real run_all.sh against stand-ins, and run_all.sh's quiet_gate called `ps -axo comm=,args=` unconditionally. On a host without ps (the lab's CLOUD task image) the script exited 127 at that line and test_benchmark_harness failed there: test_absent_pynecore_harness_completes_without_pynecore_outputs and test_absent_harness_without_committed_indicator_output_skips_three_way_report ("AssertionError: 127 != 0 ... ps: command not found").
quiet_gate now checks `command -v ps`. With ps, nothing changes: the same count, the same log line, the same tsv columns, the same report. Without ps the busy count is UNKNOWN, never 0: the timing-batch log line says "busy count unknown (ps absent)", the busy column of _workdir/speed_loads.tsv holds "unknown (ps absent)", and every figure of that run is labelled in the generated report (the line "quiet-unverified: no ps on this host, the busy count was not measured; do not use these figures as performance evidence" is appended to _workdir/speed.md and printed again as the last line of the run). With QUIET_LOAD_MAX set the wait loop decides on load alone and still ends by QUIET_WAIT_S; the label still applies. When PyneCore timings exist and the notice is set, aggregate.py writes to _workdir/speed.md instead of the committed benchmarks/results/speed.md, so unverified figures never replace the committed table. The header comment of run_all.sh says what happens without ps.
Tests (benchmarks/tests/test_run_all.py): the report cases now stub sysctl and ps so they no longer depend on the host's ps; three new cases: test_quiet_gate_without_ps_labels_figures and test_quiet_gate_without_ps_uses_load_only run run_all.sh with a PATH of symlinks that has no ps (exit 0, the log line, the tsv column, the label in speed.md and as the last output line, the committed speed.md byte-unchanged, with and without the PyneCore harness); test_quiet_gate_with_ps_preserves_verified_output runs with a ps stand-in and asserts the old output exactly and that none of the new text appears. Red first: the three new cases against the base run_all.sh fail with exit 127 and "ps: command not found" (4 subtest failures in the two missing-ps cases; the with-ps case passes on the base, as it should); on this tree the whole file passes (11 tests).
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>1 parent e11d9fc commit 9372b11
2 files changed
Lines changed: 123 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| |||
297 | 298 | | |
298 | 299 | | |
299 | 300 | | |
| 301 | + | |
| 302 | + | |
300 | 303 | | |
301 | | - | |
| 304 | + | |
302 | 305 | | |
303 | 306 | | |
304 | 307 | | |
305 | 308 | | |
306 | | - | |
307 | | - | |
308 | | - | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
309 | 319 | | |
310 | | - | |
| 320 | + | |
| 321 | + | |
311 | 322 | | |
312 | 323 | | |
313 | 324 | | |
| |||
316 | 327 | | |
317 | 328 | | |
318 | 329 | | |
319 | | - | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
320 | 335 | | |
321 | 336 | | |
322 | 337 | | |
| |||
368 | 383 | | |
369 | 384 | | |
370 | 385 | | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
371 | 390 | | |
372 | 391 | | |
373 | 392 | | |
374 | 393 | | |
375 | 394 | | |
376 | 395 | | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
377 | 399 | | |
378 | 400 | | |
379 | 401 | | |
| |||
392 | 414 | | |
393 | 415 | | |
394 | 416 | | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
212 | 218 | | |
213 | 219 | | |
214 | 220 | | |
| |||
308 | 314 | | |
309 | 315 | | |
310 | 316 | | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
311 | 403 | | |
312 | 404 | | |
313 | 405 | | |
| |||
0 commit comments