Skip to content

Commit a82902c

Browse files
authored
ci: right-size runner registration caps (#97119)
1 parent aca905c commit a82902c

4 files changed

Lines changed: 28 additions & 27 deletions

File tree

.agents/skills/openclaw-ci-limits/SKILL.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,13 @@ registration edge limit.
1313

1414
- The scarce resource is Blacksmith runner registrations, not Blacksmith vCPU
1515
capacity.
16-
- GitHub runner registrations are capped at 1,500 per 5 minutes per repository,
17-
organization, or enterprise. The `openclaw` organization shares one bucket.
16+
- GitHub runner registrations for `openclaw` are currently capped at 3,000 per
17+
5 minutes per repository, organization, or enterprise. The `openclaw`
18+
organization shares one bucket.
1819
- Core REST quota does not draw down this bucket. Check
1920
`actions_runner_registration` separately; core quota can be healthy while
2021
runner registration is throttled.
21-
- Use 1,000 registrations per 5 minutes as the operating target. Leave the last
22+
- Use 2,000 registrations per 5 minutes as the operating target. Leave the last
2223
third for other repos, retries, and burst overlap.
2324
- Jobs that route, notify, summarize, choose shards, or run short CodeQL quality
2425
scans should stay on GitHub-hosted runners unless measured evidence says
@@ -87,7 +88,7 @@ admission. The debounce only suppresses pushes that arrive while
8788
registrations are spent even if a later push cancels the run. If timing is
8889
uncertain, count every sequential push in the window.
8990

90-
Reject a change unless the org-level worst case stays below 1,000 registrations
91+
Reject a change unless the org-level worst case stays below 2,000 registrations
9192
per 5 minutes with headroom for ClawSweeper, ClawHub, Clownfish, OpenClaw RTT,
9293
and Clawbench.
9394

@@ -127,8 +128,8 @@ These are intentionally guarded by `test/scripts/ci-workflow-guards.test.ts`:
127128
- `runner-admission` on `ubuntu-24.04` with
128129
`OPENCLAW_MAIN_CI_DEBOUNCE_SECONDS=90`.
129130
- `preflight` and `security-fast` needing `runner-admission`.
130-
- CI matrix caps: fast/check lanes at 8, compact Node PR plan at current caps,
131-
Windows and Android at 2.
131+
- CI matrix caps: fast/check lanes at 12, Node test shards at 24, Windows and
132+
Android at 2.
132133
- `build-artifacts` on `blacksmith-16vcpu-ubuntu-2404`.
133134
- lower-weight Node/check shards on `blacksmith-4vcpu-ubuntu-2404`.
134135
- heavy retained Linux/Android shards on `blacksmith-8vcpu-ubuntu-2404`.

.github/workflows/ci.yml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -858,7 +858,7 @@ jobs:
858858
timeout-minutes: 60
859859
strategy:
860860
fail-fast: false
861-
max-parallel: 8
861+
max-parallel: 12
862862
matrix: ${{ fromJson(needs.preflight.outputs.checks_fast_core_matrix) }}
863863
steps:
864864
- name: Checkout
@@ -977,7 +977,7 @@ jobs:
977977
timeout-minutes: 60
978978
strategy:
979979
fail-fast: false
980-
max-parallel: 8
980+
max-parallel: 12
981981
matrix: ${{ fromJson(needs.preflight.outputs.plugin_contracts_matrix) }}
982982
steps:
983983
- name: Checkout
@@ -1058,7 +1058,7 @@ jobs:
10581058
timeout-minutes: 60
10591059
strategy:
10601060
fail-fast: false
1061-
max-parallel: 8
1061+
max-parallel: 12
10621062
matrix: ${{ fromJson(needs.preflight.outputs.channel_contracts_matrix) }}
10631063
steps:
10641064
- name: Checkout
@@ -1212,8 +1212,8 @@ jobs:
12121212
strategy:
12131213
fail-fast: false
12141214
# The canonical main path waits for the admission debounce above, so
1215-
# modestly widen this large matrix without recreating registration bursts.
1216-
max-parallel: 16
1215+
# widen this large matrix within the current runner-registration budget.
1216+
max-parallel: 24
12171217
matrix: ${{ fromJson(needs.preflight.outputs.checks_node_core_nondist_matrix) }}
12181218
steps:
12191219
- name: Checkout
@@ -1351,7 +1351,7 @@ jobs:
13511351
timeout-minutes: 20
13521352
strategy:
13531353
fail-fast: false
1354-
max-parallel: 8
1354+
max-parallel: 12
13551355
matrix:
13561356
include:
13571357
- check_name: check-guards
@@ -1493,7 +1493,7 @@ jobs:
14931493
timeout-minutes: 20
14941494
strategy:
14951495
fail-fast: false
1496-
max-parallel: 8
1496+
max-parallel: 12
14971497
matrix:
14981498
include:
14991499
- check_name: check-additional-boundaries-a

docs/ci.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,9 @@ Scope logic lives in `scripts/ci-changed-scope.mjs` and is covered by unit tests
9090

9191
The slowest Node test families are split or balanced so each job stays small without over-reserving runners: plugin contracts and channel contracts each run as two weighted Blacksmith-backed shards with the standard GitHub runner fallback, core unit fast/support lanes run separately, core runtime infra is split between state, process/config, shared, and three cron domain shards, auto-reply runs as balanced workers (with the reply subtree split into agent-runner, dispatch, and commands/state-routing shards), and agentic gateway/server configs are split across chat/auth/model/http-plugin/runtime/startup lanes instead of waiting on built artifacts. Normal CI then packs only isolated infra include-pattern shards into deterministic bundles of at most 64 test files, reducing the Node matrix without merging non-isolated command/cron, stateful agents-core, or gateway/server suites; heavy fixed suites stay on 8 vCPU while the bundled and lower-weight lanes use 4 vCPU. Pull requests on the canonical repository use an additional compact admission plan: the same per-config groups run in isolated subprocesses inside the current 34-job Linux Node plan, so a single PR does not register the full 70-plus-job Node matrix. `main` pushes, manual dispatches, and release gates retain the full matrix. Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so `.artifacts/vitest-shard-timings.json` can distinguish a whole config from a filtered shard. `check-additional-*` keeps package-boundary compile/canary work together and separates runtime topology architecture from gateway watch coverage; the boundary guard list is striped into one prompt-heavy shard and one combined shard for the remaining guard stripes, each running selected independent guards concurrently and printing per-check timings. The expensive Codex happy-path prompt snapshot drift check runs as its own additional job for manual CI and for prompt-affecting changes only, so normal unrelated Node changes do not wait behind cold prompt snapshot generation and the boundary shards stay balanced while prompt drift is still pinned to the PR that caused it; the same flag skips prompt snapshot Vitest generation inside the built-artifact core support-boundary shard. Gateway watch, channel tests, and the core support-boundary shard run concurrently inside `build-artifacts` after `dist/` and `dist-runtime/` are already built.
9292

93-
Once admitted, canonical Linux CI permits up to 12 concurrent Node jobs and 8 for
94-
the smaller fast/check lanes; Windows and Android stay at two because those
95-
runner pools are narrower.
93+
Once admitted, canonical Linux CI permits up to 24 concurrent Node test jobs and
94+
12 for the smaller fast/check lanes; Windows and Android stay at two because
95+
those runner pools are narrower.
9696

9797
The compact PR plan emits 18 Node jobs for the current suite: whole-config
9898
groups are batched in isolated subprocesses with a 120-minute batch timeout,
@@ -145,17 +145,17 @@ gh workflow run full-release-validation.yml --ref main -f ref=<branch-or-sha>
145145

146146
## Runner registration budget
147147

148-
GitHub caps self-hosted runner registrations at 1,500 runners per 5 minutes per
149-
repository, organization, or enterprise. The limit is shared by all Blacksmith
150-
runner registrations in the `openclaw` organization, so adding another
151-
Blacksmith installation does not add a new bucket.
148+
OpenClaw's current GitHub runner-registration bucket allows 3,000 self-hosted
149+
runner registrations per 5 minutes. The limit is shared by all Blacksmith runner
150+
registrations in the `openclaw` organization, so adding another Blacksmith
151+
installation does not add a new bucket.
152152

153153
Treat Blacksmith labels as the scarce resource for burst control. Jobs that
154154
only route, notify, summarize, select shards, or run short CodeQL scans should
155155
stay on GitHub-hosted runners unless they have measured Blacksmith-specific
156156
needs. Any new Blacksmith matrix, larger `max-parallel`, or high-frequency
157157
workflow must show its worst-case registration count and keep the org-level
158-
target below 1,000 registrations per 5 minutes, leaving headroom for concurrent
158+
target below 2,000 registrations per 5 minutes, leaving headroom for concurrent
159159
repositories and retried jobs.
160160

161161
Canonical-repo CI keeps Blacksmith as the default runner path for normal push and pull-request runs. `workflow_dispatch` and non-canonical repository runs use GitHub-hosted runners, but normal canonical runs do not currently probe Blacksmith queue health or automatically fall back to GitHub-hosted labels when Blacksmith is unavailable.

test/scripts/ci-workflow-guards.test.ts

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -182,12 +182,12 @@ describe("ci workflow guards", () => {
182182
expect(workflow.concurrency["cancel-in-progress"]).toContain(
183183
"github.event_name == 'pull_request'",
184184
);
185-
expect(workflow.jobs["checks-fast-core"].strategy["max-parallel"]).toBe(8);
186-
expect(workflow.jobs["checks-node-core-test-nondist-shard"].strategy["max-parallel"]).toBe(16);
187-
expect(workflow.jobs["checks-fast-plugin-contracts-shard"].strategy["max-parallel"]).toBe(8);
188-
expect(workflow.jobs["checks-fast-channel-contracts-shard"].strategy["max-parallel"]).toBe(8);
189-
expect(workflow.jobs["check-shard"].strategy["max-parallel"]).toBe(8);
190-
expect(workflow.jobs["check-additional-shard"].strategy["max-parallel"]).toBe(8);
185+
expect(workflow.jobs["checks-fast-core"].strategy["max-parallel"]).toBe(12);
186+
expect(workflow.jobs["checks-node-core-test-nondist-shard"].strategy["max-parallel"]).toBe(24);
187+
expect(workflow.jobs["checks-fast-plugin-contracts-shard"].strategy["max-parallel"]).toBe(12);
188+
expect(workflow.jobs["checks-fast-channel-contracts-shard"].strategy["max-parallel"]).toBe(12);
189+
expect(workflow.jobs["check-shard"].strategy["max-parallel"]).toBe(12);
190+
expect(workflow.jobs["check-additional-shard"].strategy["max-parallel"]).toBe(12);
191191
expect(workflow.jobs["checks-windows"].strategy["max-parallel"]).toBe(2);
192192
expect(workflow.jobs.android.strategy["max-parallel"]).toBe(2);
193193
});

0 commit comments

Comments
 (0)