Skip to content

Commit 2aab17c

Browse files
steipetesteipete
committed
perf(ci): rebalance compact node test shards from fresh timing census
Recalibrate COMPACT_GROUP_SECONDS_HINTS and the cli-runner stripe file hints against mean [shard:*] begin->end wall clock from five green Blacksmith compact PR runs (29557851276, 29558164241, 29558528472, 29558634980, 29558677406). Measured before (job wall, 5-run mean): 26 jobs; tail compact-large-6 = 317s (cli-runner stripe with the whale file) vs siblings 139-290s; compact-small-10 = 277s. Key drifts: cli-runner.reliability.test.ts measures 245s solo (import-dominated; stripe hint said 7s, so LPT packed it with extra groups), auto-reply commands stripes 26/9/12s vs 92s hints, core-unit-fast 142s vs 190s, tui-pty 103s vs 200s. Computed after (planner output, measured-mean walls): 23 jobs (-4 fixed ~60s setups); reliability stripe isolated as its own 305s bin (floor set by that single file's 245s module import), every sibling bin <= 280s; large-runner class max/mean = 1.145. Pinned-worker and exclusive-bin invariants unchanged.
1 parent 8fa4867 commit 2aab17c

1 file changed

Lines changed: 94 additions & 84 deletions

File tree

‎scripts/lib/ci-node-test-plan.mjs‎

Lines changed: 94 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -36,101 +36,111 @@ const COMPACT_TOOLING_NODE_TEST_GROUPS = 4;
3636
const COMPACT_WHOLE_NODE_TEST_TIMEOUT_MINUTES = 120;
3737
const AUTO_REPLY_COMMANDS_STRIPES = 3;
3838
const AGENTS_CORE_RUNNER_CLI_STRIPES = 3;
39-
// Advisory runtime estimates (seconds) per split shard, measured from a
40-
// Blacksmith compact PR run with serial plans and 2 Vitest workers (run
41-
// 29481835688). Packing only: a stale entry skews job balance but never
42-
// correctness. Unknown shards fall back to a per-file estimate.
39+
// Advisory runtime estimates (seconds) per split shard: mean [shard:*]
40+
// begin->end wall clock across five green Blacksmith compact PR runs
41+
// (29557851276, 29558164241, 29558528472, 29558634980, 29558677406).
42+
// Packing only: a stale entry skews job balance but never correctness.
43+
// Unknown shards fall back to a per-file estimate.
4344
const COMPACT_GROUP_SECONDS_HINTS = new Map([
44-
["agentic-agents-core-auth", 32],
45-
["agentic-agents-core-isolated", 10],
46-
["agentic-agents-core-models", 66],
47-
// Cost-aware striping balances the cli-runner and commands stripes; keep
48-
// their hints equalized at the old totals split evenly.
49-
["agentic-agents-core-runner-cli-1", 103],
50-
["agentic-agents-core-runner-cli-2", 103],
51-
["agentic-agents-core-runner-cli-3", 103],
52-
["agentic-agents-core-runner-commands", 30],
53-
["agentic-agents-core-runner-embedded", 20],
54-
["agentic-agents-core-runner-sessions", 18],
55-
["agentic-agents-core-runtime", 81],
56-
["agentic-agents-core-subagents", 37],
57-
["agentic-agents-core-tools", 55],
58-
["agentic-agents-embedded", 59],
59-
["agentic-agents-support", 88],
60-
["agentic-agents-tools", 43],
61-
["agentic-cli", 81],
62-
["agentic-command-support", 28],
63-
["agentic-commands-agent-channel", 47],
64-
["agentic-commands-doctor", 23],
65-
["agentic-commands-doctor-auth", 10],
66-
["agentic-commands-doctor-config-state", 41],
67-
["agentic-commands-doctor-gateway", 7],
68-
["agentic-commands-doctor-plugins-tools", 10],
45+
["agentic-agents-core-auth", 23],
46+
["agentic-agents-core-isolated", 8],
47+
["agentic-agents-core-models", 60],
48+
// cli-runner.reliability.test.ts imports a ~245s module graph on its own
49+
// (79 tests in <2s, import dominates); one file cannot be split further, so
50+
// its stripe is a near-solo bin and the sibling stripes stay tiny.
51+
["agentic-agents-core-runner-cli-1", 245],
52+
["agentic-agents-core-runner-cli-2", 18],
53+
["agentic-agents-core-runner-cli-3", 8],
54+
["agentic-agents-core-runner-commands", 25],
55+
["agentic-agents-core-runner-embedded", 9],
56+
["agentic-agents-core-runner-sessions", 10],
57+
["agentic-agents-core-runtime", 67],
58+
["agentic-agents-core-subagents", 25],
59+
["agentic-agents-core-tools", 50],
60+
["agentic-agents-embedded", 54],
61+
["agentic-agents-support", 108],
62+
["agentic-agents-tools", 41],
63+
["agentic-cli", 68],
64+
["agentic-command-support", 39],
65+
["agentic-commands-agent-channel", 56],
66+
["agentic-commands-doctor", 10],
67+
["agentic-commands-doctor-auth", 5],
68+
["agentic-commands-doctor-config-state", 37],
69+
["agentic-commands-doctor-gateway", 6],
70+
["agentic-commands-doctor-plugins-tools", 9],
6971
["agentic-commands-doctor-sessions-cron", 24],
70-
["agentic-commands-models", 21],
71-
["agentic-commands-onboard-config", 15],
72-
["agentic-commands-status-tools", 30],
73-
["agentic-control-plane-agent-chat", 75],
74-
["agentic-control-plane-auth-node", 101],
75-
["agentic-control-plane-http-models", 40],
76-
["agentic-control-plane-http-plugin-ws", 43],
77-
["agentic-control-plane-runtime-config", 28],
78-
["agentic-control-plane-runtime-cron", 29],
79-
["agentic-control-plane-runtime-server", 26],
72+
["agentic-commands-doctor-shared", 13],
73+
["agentic-commands-models", 14],
74+
["agentic-commands-onboard-config", 10],
75+
["agentic-commands-status-tools", 25],
76+
["agentic-control-plane-agent-chat", 71],
77+
["agentic-control-plane-auth-node", 86],
78+
["agentic-control-plane-http-models", 20],
79+
["agentic-control-plane-http-plugin-ws", 36],
80+
["agentic-control-plane-runtime-config", 12],
81+
["agentic-control-plane-runtime-cron", 25],
82+
["agentic-control-plane-runtime-server", 14],
83+
["agentic-control-plane-runtime-shared-token", 17],
8084
["agentic-control-plane-runtime-state", 25],
81-
["agentic-control-plane-runtime-ui-tools", 24],
82-
["agentic-control-plane-startup-core", 113],
83-
["agentic-control-plane-startup-health-runtime", 24],
84-
["agentic-control-plane-startup-restart-close", 20],
85-
["agentic-gateway-core", 130],
86-
["agentic-gateway-methods", 79],
87-
["agentic-plugin-sdk", 50],
88-
["auto-reply-core-top-level", 33],
89-
["auto-reply-reply-agent-runner", 52],
90-
["auto-reply-reply-commands-1", 92],
91-
["auto-reply-reply-commands-2", 92],
92-
["auto-reply-reply-commands-3", 92],
93-
["auto-reply-reply-dispatch", 36],
94-
["auto-reply-reply-session", 21],
95-
["auto-reply-reply-state-routing", 21],
96-
["core-runtime-cron-core", 24],
97-
["core-runtime-cron-isolated-agent", 61],
98-
["core-runtime-cron-service", 24],
99-
["core-runtime-hooks", 11],
100-
["core-runtime-infra-approval-exec", 30],
101-
["core-runtime-infra-channel-plugin", 26],
102-
["core-runtime-infra-heartbeat-runner", 77],
103-
["core-runtime-infra-net-install", 14],
104-
["core-runtime-infra-outbound-actions", 21],
105-
["core-runtime-infra-outbound-core", 44],
106-
["core-runtime-infra-process", 83],
107-
["core-runtime-infra-provider-push", 20],
108-
["core-runtime-infra-storage-state", 58],
109-
["core-runtime-infra-system-runtime", 40],
110-
["core-runtime-media-ui", 131],
111-
["core-runtime-secrets", 43],
112-
["core-runtime-shared", 48],
113-
// PTY timing tests inflate badly next to co-runners; keep this group in a
114-
// lightly packed bin so its lane stays close to solo runtime.
115-
["core-runtime-tui-pty", 200],
116-
["core-tooling-1", 113],
117-
["core-tooling-2", 113],
118-
["core-tooling-3", 113],
119-
["core-tooling-4", 113],
85+
["agentic-control-plane-runtime-ui-tools", 10],
86+
["agentic-control-plane-startup-core", 98],
87+
["agentic-control-plane-startup-health-runtime", 9],
88+
["agentic-control-plane-startup-restart-close", 16],
89+
["agentic-gateway-core", 127],
90+
["agentic-gateway-methods", 68],
91+
["agentic-plugin-sdk", 46],
92+
["auto-reply-core-top-level", 29],
93+
["auto-reply-reply-agent-runner", 35],
94+
["auto-reply-reply-commands-1", 26],
95+
["auto-reply-reply-commands-2", 9],
96+
["auto-reply-reply-commands-3", 12],
97+
["auto-reply-reply-dispatch", 40],
98+
["auto-reply-reply-session", 19],
99+
["auto-reply-reply-state-routing", 18],
100+
["core-runtime-cron-core", 19],
101+
["core-runtime-cron-isolated-agent", 51],
102+
["core-runtime-cron-service", 20],
103+
["core-runtime-hooks", 8],
104+
["core-runtime-infra-approval-exec", 27],
105+
["core-runtime-infra-channel-plugin", 16],
106+
["core-runtime-infra-diagnostics-state", 14],
107+
["core-runtime-infra-heartbeat-runner", 51],
108+
["core-runtime-infra-misc", 11],
109+
["core-runtime-infra-net-install", 10],
110+
["core-runtime-infra-outbound-actions", 17],
111+
["core-runtime-infra-outbound-core", 39],
112+
["core-runtime-infra-process", 89],
113+
["core-runtime-infra-provider-push", 14],
114+
["core-runtime-infra-storage-state", 55],
115+
["core-runtime-infra-system-runtime", 35],
116+
["core-runtime-media-ui", 113],
117+
["core-runtime-secrets", 45],
118+
["core-runtime-shared", 46],
119+
// PTY timing suites still need a lightly packed lane; the exclusive-bin cap
120+
// leaves only trivial co-groups next to this measured runtime.
121+
["core-runtime-tui-pty", 103],
122+
["core-tooling-1", 85],
123+
["core-tooling-2", 112],
124+
["core-tooling-3", 106],
125+
["core-tooling-4", 105],
120126
["core-tooling-docker", 6],
121-
["core-tooling-isolated", 55],
122-
["core-unit-fast", 190],
123-
["core-unit-src-security", 138],
124-
["core-unit-support", 19],
127+
["core-tooling-isolated", 52],
128+
["core-unit-fast", 142],
129+
["core-unit-src-security", 108],
130+
["core-unit-support", 15],
125131
]);
126132
// Advisory per-file wall-clock hints (seconds) for stripe balancing, measured
127133
// from single-file local runs (M4 Max) and static import-graph size. Packing
128134
// only: a stale entry skews stripe balance but never correctness. Unlisted
129135
// files use the default, which mostly reflects the per-file module-graph
130136
// re-evaluation cost that dominates these serial suites.
131137
const STRIPE_FILE_SECONDS_HINTS = new Map([
132-
["src/agents/cli-runner.reliability.test.ts", 7],
133-
["src/agents/cli-runner.spawn.test.ts", 8],
138+
// cli-runner entries are CI wall clock (begin->checkmark deltas from the
139+
// compact runs above); reliability's import graph alone runs ~245s, so LPT
140+
// must isolate it or its stripe owns the PR tail.
141+
["src/agents/cli-runner.context-engine.test.ts", 6],
142+
["src/agents/cli-runner.reliability.test.ts", 245],
143+
["src/agents/cli-runner.spawn.test.ts", 18],
134144
["src/auto-reply/reply/commands-export-session.test.ts", 8],
135145
["src/auto-reply/reply/commands-gating.test.ts", 6],
136146
["src/auto-reply/reply/commands-learn.test.ts", 8],

0 commit comments

Comments
 (0)