perf(ci): correct pack hints from measured stripe walls#109899
Merged
Conversation
Group walls measured from compact run 29564411446 (post-#109769): core-unit-fast-isolated runs 78.6s on 4 vCPU because fork-per-file isolation parallelizes poorly there; pin it to the 8 vCPU class where the same segment runs ~50s. Unit-fast stripes measure ~25-37s (hinted 100/60) and tooling stripes 71-87s; refresh both so the packer stops overfilling small bins (the 299s compact tail this run produced). Cheap stripes may now co-locate in one bin, so the compact test only requires their presence.
openclaw-barnacle
Bot
added
scripts
clawsweeper
Bot
added
rating: 🐚 platinum hermit
Contributor
|
Codex review: needs real behavior proof before merge. Reviewed July 17, 2026, 6:02 AM ET / 10:02 UTC. Summary PR surface: Tests -1, Other +5. Total +4 across 2 files. Reproducibility: yes. at the evidence level: the predecessor workflow timestamps identify the overfilled bin and quantify the hint mismatch, though this review could not independently replay that historical run. Review metrics: none identified. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Keep the canonical cost-aware packer and merge the measured hint update after exact-head compact CI confirms the isolated shard uses 8 vCPU and the resulting bin walls remain near the target. Do we have a high-confidence way to reproduce the issue? Yes at the evidence level: the predecessor workflow timestamps identify the overfilled bin and quantify the hint mismatch, though this review could not independently replay that historical run. Is this the best way to solve the issue? Yes. Updating the canonical measured hints and runner-class exception is narrower and more maintainable than changing shard topology or introducing a second packing path. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 44f20956e330. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Tests -1, Other +5. Total +4 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
Contributor
Author
|
Merged via squash.
|
steipete
added a commit
that referenced
this pull request
Jul 17, 2026
…walls Second calibration pass after the cli-runner reliability whale fix (#109772) and the stripe-wall correction (#109899): the compact group hints drifted below loaded-fleet reality, so FFD packed the heaviest groups into ~300s bins (checks-node-compact-large-2 at 242-314s shard wall, up to 389s job wall) while tail bins idled. Method: parsed [shard:*] begin/end timestamps from all 147 compact job logs across seven green CI runs whose head SHAs carry the whale-fix hints (runs 29605136624, 29605203485, 29605983019, 29606701461, 29611308972, 29611457693, 29611500865; 7 samples per group). Each hint is the per-group mean after dropping cache-warm/contention outliers outside [median/1.5, median*1.5]. Packing cap: with honest loaded walls a 220s cap no longer fits either pool (adds one job to each); 235s keeps the same job count (6 large + 15 small + dist) and flattens the ceiling. Median job setup overhead measured 60s (p90 87s), so a 235s bin stays near the 5-minute PR budget. Predicted bin shard walls (sum of measured group means): before: large 220/301/200/219/257/112, small heavies 251/235/218/217/215, max 301 after: large 235/232/229/228/228/156, small heavies 236/234/234/234/233, max 236 large-pool max/mean 1.38 -> 1.08 Biggest hint deltas (old -> new): startup-core 98->156, core-tooling-4 71->125, core-unit-fast-isolated 50->90, core-tooling-3 82->108, runtime-server 14->29, storage-state 55->70, media-ui 113->124, tui-pty 103->116, runner-cli-1 18->8, unit-src-security 108->95. Exclusive bins keep their 150s cap and 5-bin count: tui-pty (116) no longer shares a bin with tooling-isolated (166s combined measured); FFD now pairs tooling-2 with tooling-isolated at 144s. STRIPE_FILE_SECONDS_HINTS left unchanged: measured stripe walls sit in exclusive bins far below the packing ceiling (tooling stripes 94-125s), and refitting per-file hints would reshuffle stripe membership and invalidate the measured group means above.
steipete
added a commit
that referenced
this pull request
Jul 17, 2026
…walls (#110172) Second calibration pass after the cli-runner reliability whale fix (#109772) and the stripe-wall correction (#109899): the compact group hints drifted below loaded-fleet reality, so FFD packed the heaviest groups into ~300s bins (checks-node-compact-large-2 at 242-314s shard wall, up to 389s job wall) while tail bins idled. Method: parsed [shard:*] begin/end timestamps from all 147 compact job logs across seven green CI runs whose head SHAs carry the whale-fix hints (runs 29605136624, 29605203485, 29605983019, 29606701461, 29611308972, 29611457693, 29611500865; 7 samples per group). Each hint is the per-group mean after dropping cache-warm/contention outliers outside [median/1.5, median*1.5]. Packing cap: with honest loaded walls a 220s cap no longer fits either pool (adds one job to each); 235s keeps the same job count (6 large + 15 small + dist) and flattens the ceiling. Median job setup overhead measured 60s (p90 87s), so a 235s bin stays near the 5-minute PR budget. Predicted bin shard walls (sum of measured group means): before: large 220/301/200/219/257/112, small heavies 251/235/218/217/215, max 301 after: large 235/232/229/228/228/156, small heavies 236/234/234/234/233, max 236 large-pool max/mean 1.38 -> 1.08 Biggest hint deltas (old -> new): startup-core 98->156, core-tooling-4 71->125, core-unit-fast-isolated 50->90, core-tooling-3 82->108, runtime-server 14->29, storage-state 55->70, media-ui 113->124, tui-pty 103->116, runner-cli-1 18->8, unit-src-security 108->95. Exclusive bins keep their 150s cap and 5-bin count: tui-pty (116) no longer shares a bin with tooling-isolated (166s combined measured); FFD now pairs tooling-2 with tooling-isolated at 144s. STRIPE_FILE_SECONDS_HINTS left unchanged: measured stripe walls sit in exclusive bins far below the packing ceiling (tooling stripes 94-125s), and refitting per-file hints would reshuffle stripe membership and invalidate the measured group means above.
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jul 18, 2026
) Group walls measured from compact run 29564411446 (post-openclaw#109769): core-unit-fast-isolated runs 78.6s on 4 vCPU because fork-per-file isolation parallelizes poorly there; pin it to the 8 vCPU class where the same segment runs ~50s. Unit-fast stripes measure ~25-37s (hinted 100/60) and tooling stripes 71-87s; refresh both so the packer stops overfilling small bins (the 299s compact tail this run produced). Cheap stripes may now co-locate in one bin, so the compact test only requires their presence.
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jul 18, 2026
…walls (openclaw#110172) Second calibration pass after the cli-runner reliability whale fix (openclaw#109772) and the stripe-wall correction (openclaw#109899): the compact group hints drifted below loaded-fleet reality, so FFD packed the heaviest groups into ~300s bins (checks-node-compact-large-2 at 242-314s shard wall, up to 389s job wall) while tail bins idled. Method: parsed [shard:*] begin/end timestamps from all 147 compact job logs across seven green CI runs whose head SHAs carry the whale-fix hints (runs 29605136624, 29605203485, 29605983019, 29606701461, 29611308972, 29611457693, 29611500865; 7 samples per group). Each hint is the per-group mean after dropping cache-warm/contention outliers outside [median/1.5, median*1.5]. Packing cap: with honest loaded walls a 220s cap no longer fits either pool (adds one job to each); 235s keeps the same job count (6 large + 15 small + dist) and flattens the ceiling. Median job setup overhead measured 60s (p90 87s), so a 235s bin stays near the 5-minute PR budget. Predicted bin shard walls (sum of measured group means): before: large 220/301/200/219/257/112, small heavies 251/235/218/217/215, max 301 after: large 235/232/229/228/228/156, small heavies 236/234/234/234/233, max 236 large-pool max/mean 1.38 -> 1.08 Biggest hint deltas (old -> new): startup-core 98->156, core-tooling-4 71->125, core-unit-fast-isolated 50->90, core-tooling-3 82->108, runtime-server 14->29, storage-state 55->70, media-ui 113->124, tui-pty 103->116, runner-cli-1 18->8, unit-src-security 108->95. Exclusive bins keep their 150s cap and 5-bin count: tui-pty (116) no longer shares a bin with tooling-isolated (166s combined measured); FFD now pairs tooling-2 with tooling-isolated at 144s. STRIPE_FILE_SECONDS_HINTS left unchanged: measured stripe walls sit in exclusive bins far below the packing ceiling (tooling stripes 94-125s), and refitting per-file hints would reshuffle stripe membership and invalidate the measured group means above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
First measured feedback loop after #109769: the compact PR plan's packing tail dropped 419s -> 299s, but the new tail bin (
compact-small-11) was overfilled becausecore-unit-fast-isolatedactually costs 78.6s on a 4 vCPU runner against a 25s hint — fork-per-file isolation parallelizes poorly on the small class. Unit-fast stripes also measured far cheaper than hinted (~25-37s vs 100/60) and tooling stripes measured 71-87s.Why This Change Was Made
Hint drift directly sets PR wall time now that packing is cost-aware; these are the first corrections from real post-landing group walls (parsed from
[shard:*]timestamps in compact run 29564411446).User Impact
Compact PR bins repack closer to the 220s target: the isolated shard moves to the 8 vCPU class (measured ~50s there), and honest stripe hints stop the packer from overfilling small bins. Expected compact tail: ~250s -> ~230s and better balance; no coverage or topology changes.
Evidence
core-unit-fast-isolated78.6s (4 vCPU),core-unit-fast-236.7s,core-tooling-1..487/80/82/71s,core-tooling-isolated44.6s.core-unit-fast-isolatedjoinsKEEP_LARGE_NODE_TEST_RUNNER; verified both plans now route it toblacksmith-8vcpu(full-plan shard runner and compact bin class).