Skip to content

Commit abb6f04

Browse files
vincentkocvincentkoc
authored
ci(release): harden release controls
One-time maintainer-authorized bootstrap merge for the release-gate verifier policy. Exact hosted CI and all supporting workflow gates passed on 66133de.
1 parent 3a570f1 commit abb6f04

88 files changed

Lines changed: 3403 additions & 449 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/release-openclaw-ci/SKILL.md‎

Lines changed: 37 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,25 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
2424
fails, the parent cancels the remaining child matrix and prints the failed
2525
job summary. Inspect that first red job instead of waiting for unrelated
2626
matrix tails.
27+
- In a sparse worktree or Testbox source sync, first confirm `package.json`,
28+
`pnpm-lock.yaml`, and every source path the selected check reads. If any are
29+
absent, that checkout cannot validate a release dependency or Docker lane:
30+
stop and use the repo remote changed gate or a full task worktree. When the
31+
inputs are present and a release fix changes `package.json` or
32+
`pnpm-lock.yaml`, rebuild only the task-owned disposable box with
33+
`CI=true pnpm install --frozen-lockfile`, then run an explicit
34+
`require.resolve()` probe before Docker or focused tests. The CI flag permits
35+
pnpm to recreate a prewarmed modules directory without an interactive
36+
confirmation. Do not weaken the lockfile or label sparse-checkout failures
37+
as product/Docker failures.
38+
- If the candidate is rebased or its base SHA changes after warmup, stop the
39+
task-owned box and warm a fresh one before testing. Testbox source sync is
40+
relative to the warmed source tree; continuing can mix an old base file with
41+
a new candidate diff and produce false lockfile or Docker failures.
42+
- For a committed release candidate, warm the box with
43+
`blacksmith testbox warmup ... --ref <candidate-branch-or-sha>`. Do not rely
44+
on source sync to overlay committed branch changes onto the workflow's
45+
default ref.
2746

2847
## Preflight
2948

@@ -57,7 +76,7 @@ gh workflow run openclaw-performance.yml \
5776
-f repeat=3 \
5877
-f deep_profile=false \
5978
-f live_openai_candidate=false \
60-
-f fail_on_regression=false
79+
-f fail_on_regression=true
6180
```
6281

6382
- Do not wait for full release validation to start this early perf signal.
@@ -66,8 +85,9 @@ gh workflow run openclaw-performance.yml \
6685
- Call out any regression in the release proof. Treat a major regression as a
6786
release blocker until it is fixed, waived by the operator, or proven to be
6887
infrastructure noise.
69-
- Full Release Validation also records advisory product-performance evidence;
70-
the early standalone run is for overlap and faster regression discovery.
88+
- Full Release Validation records blocking product-performance evidence. The
89+
early standalone run is for overlap and faster regression discovery, but a
90+
regression or missing child run blocks the parent validation.
7191

7292
Prefer the trusted workflow on `main`, target the exact release SHA:
7393

@@ -89,7 +109,7 @@ gh workflow run full-release-validation.yml \
89109
-f rerun_group=all
90110
```
91111

92-
Use `release_profile=stable` unless the operator explicitly asks for the broad advisory provider/media matrix. Use narrow `rerun_group` after focused fixes.
112+
Use `release_profile=stable` unless the operator explicitly asks for the broad advisory provider/media matrix. Stable and full profiles force the release soak; the beta profile may opt in with `run_release_soak=true`. Use narrow `rerun_group` after focused fixes.
93113
Publish with `openclaw-release-publish.yml` using `release_profile=from-validation`
94114
unless a maintainer intentionally wants to cross-check a specific profile; the
95115
publish workflow reads the effective profile from the full-validation manifest.
@@ -125,6 +145,19 @@ Stop watchers before ending the turn or switching strategy.
125145
Anthropic API-key lane.
126146
5. For live-cache failures, inspect whether it is missing/invalid key, empty text, provider refusal, timeout, or baseline miss. Do not weaken release gates without clear provider evidence.
127147
6. Fix narrowly, run local/changed proof, commit, push, rerun the smallest matching group.
148+
7. If a required PR CI run is capacity-stalled with queued jobs and no active
149+
jobs, do not cancel unrelated work or accept a generic manual dispatch.
150+
From the PR head branch, dispatch the explicit exact-SHA fallback:
151+
`gh workflow run ci.yml --repo openclaw/openclaw --ref <pr-head-branch> -f
152+
target_ref=<full-pr-sha> -f include_android=true -f release_gate=true`.
153+
It runs on GitHub-hosted runners and is accepted only when its run title is
154+
`CI release gate <full-pr-sha>`. Record the stalled Blacksmith run and the
155+
fallback run in release evidence.
156+
If `Blacksmith Build Artifacts Testbox` is the only remaining required gate
157+
and remains queued without a runner, that completed exact fallback may cover
158+
it because CI's `build-artifacts` job already builds, packages, and smoke
159+
tests the artifacts. Do not use this coverage after the artifact workflow
160+
starts or completes non-successfully.
128161

129162
## Evidence
130163

‎.agents/skills/release-openclaw-maintainer/SKILL.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,10 @@ Use this skill for release and publish-time workflow. Load `$release-private` if
1717
- This skill should be sufficient to drive the normal release flow end-to-end.
1818
- Use the private maintainer release docs for credentials, recovery steps, and mac signing/notary specifics, and use `docs/reference/RELEASING.md` for public policy.
1919
- Core `openclaw` publish is manual `workflow_dispatch`; creating or pushing a tag does not publish by itself.
20+
- Do not edit the root `README.md` as release prep, release closeout, or a
21+
substitute for release notes. Package-root README validation is a hard
22+
packaging gate, but a release only changes README content when an actual
23+
user-facing documentation contract changed.
2024
- Normal release work happens on a branch cut from `main`, not directly on
2125
`main`. Use `release/YYYY.M.PATCH` for the branch name.
2226
- If the operator asks for a release without saying stable/full, default to
@@ -76,6 +80,44 @@ Use this skill for release and publish-time workflow. Load `$release-private` if
7680
or clawgrit reports. Report regressions explicitly. A major regression is a
7781
release blocker unless the operator waives it or the data clearly proves
7882
infrastructure noise.
83+
- Heal CI before tagging or publishing. The exact candidate SHA must have green
84+
`Full Release Validation`, including the root Dockerfile/install-smoke path.
85+
Treat a red Docker, package, or release workflow lane as a release-branch
86+
defect until the smallest correct fix is landed and proven; do not waive it
87+
because npm preflight or another sibling lane passed.
88+
- Keep the canonical `scripts/pr` runner authoritative for prepare and merge
89+
artifacts. A release-gate policy change may use focused candidate tests and
90+
exact-SHA hosted CI for proof, but never route `prepare-*` or `merge-*`
91+
through PR-controlled scripts or synthesize prepare artifacts to bootstrap
92+
the change. If the current canonical gate cannot validate the new policy,
93+
stop for explicit maintainer direction rather than weakening that boundary.
94+
- In maintainer Testbox mode, use `OPENCLAW_TESTBOX=1 scripts/pr prepare-run
95+
<PR>` only after the exact PR head has passed `CI` and every scheduled
96+
hosted gate. For a workflow change, that means `Blacksmith Testbox`,
97+
`Blacksmith ARM Testbox`, `Blacksmith Build Artifacts Testbox`, and
98+
`Workflow Sanity`; only gates GitHub actually scheduled for that exact head
99+
are required. This preserves the canonical prepare artifacts while avoiding
100+
a redundant broad local suite. A
101+
literal `CHANGELOG.md`-only head gets a clean diff check instead because
102+
those workflows intentionally do not dispatch. Documentation and README
103+
changes still require CI. If `merge-run` requires a mainline sync, run
104+
`OPENCLAW_TESTBOX=1 scripts/pr prepare-sync-head <PR>`, wait for those hosted
105+
gates on the newly pushed SHA, then run `prepare-run` again.
106+
- If an exact PR-head CI run has no active jobs because Blacksmith capacity is
107+
stalled, a maintainer may dispatch the explicit GitHub-hosted fallback from
108+
the PR head branch:
109+
`gh workflow run ci.yml --repo openclaw/openclaw --ref <pr-head-branch> -f
110+
target_ref=<full-pr-sha> -f include_android=true -f release_gate=true`.
111+
Use it only for an observed provider queue stall, never for failed CI or as a
112+
routine shortcut. The run must be named `CI release gate <full-pr-sha>` and
113+
pass on that exact SHA; the native hosted-gate verifier rejects generic manual
114+
CI runs. If `Blacksmith Build Artifacts Testbox` is the only remaining
115+
required gate and it is still queued without a runner, the same completed
116+
fallback CI may cover it because its `build-artifacts` job builds, packages,
117+
and smoke tests those artifacts. The verifier records that coverage. Never
118+
use this coverage when the artifact workflow has started, failed, been
119+
cancelled, or been skipped. Then rerun `OPENCLAW_TESTBOX=1 scripts/pr
120+
prepare-run <PR>`.
79121
- Generate the changelog before every beta, beta rerun, stable release, or
80122
stable rerun, before version/tag preparation. Use
81123
`$openclaw-changelog-update` for the rewrite. Do not continue release prep if
@@ -119,6 +161,14 @@ Stable publication is not complete until `main` carries the actual shipped relea
119161
`OPENCLAW_TESTBOX=1 pnpm check:changed`. Push, then verify `origin/main`
120162
contains the shipped version and changelog before calling the stable release
121163
done.
164+
6. Keep repository variables `RELEASE_ROLLBACK_DRILL_ID` and
165+
`RELEASE_ROLLBACK_DRILL_DATE` current after each private rollback drill.
166+
`openclaw-stable-main-closeout.yml` starts from the `main` push carrying the
167+
shipped version, changelog, and appcast after stable publication, then binds
168+
immutable evidence to the published tag. Do not declare stable complete
169+
until it writes the immutable closeout manifest to the GitHub release. The
170+
drill must be within 90 days; manual dispatch is only for repair/replay, and
171+
private rollback commands remain in the maintainer-only runbook.
122172

123173
## Handle versions and release files consistently
124174

‎.agents/skills/verify-release/SKILL.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,17 @@ publish skill; use `$release-openclaw-maintainer` before changing release state.
2929
- Confirm release body has npm, CI, plugin npm, ClawHub, mac/appcast evidence
3030
links when expected.
3131
- Confirm assets expected for stable mac releases are uploaded: zip, dmg,
32-
dSYM, dependency evidence when present.
32+
dSYM, dependency evidence, immutable full-validation manifest,
33+
postpublish evidence, and stable-main closeout manifest.
34+
- Download each immutable evidence asset and its `.sha256` companion, then
35+
verify the checksum before trusting the release record.
3336
2. Root npm:
3437
- `npm view openclaw@<VERSION> version dist-tags.latest dist.tarball dist.integrity time.<VERSION> --json`
3538
- `latest` must equal `<VERSION>` for stable.
3639
- Record tarball, integrity, publish time.
40+
- Confirm the release postpublish evidence records
41+
`npmRegistrySignaturesVerified: true` and
42+
`npmProvenanceAttestationMatched: true`.
3743
3. Plugin publish set:
3844
- Get exact tag metadata from GitHub, not the local checkout when dirty:
3945
download `https://api.github.com/repos/openclaw/openclaw/tarball/v<VERSION>`
@@ -57,6 +63,9 @@ publish skill; use `$release-openclaw-maintainer` before changing release state.
5763
Full Release Validation, OpenClaw Release Checks, OpenClaw NPM Release,
5864
Plugin NPM Release, Plugin ClawHub Release, mac preflight/validation/publish
5965
when stable mac assets are expected.
66+
- For stable, verify `OpenClaw Stable Main Closeout` succeeded and its
67+
manifest records the matching release tag, current rollback drill, stable
68+
soak, and blocking performance evidence.
6069
- Summarize only relevant successful/failed jobs; ignore routine skipped
6170
optional lanes unless the release body promised them.
6271
6. Published package smoke:

‎.github/actions/docker-e2e-plan/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ runs:
113113
114114
- name: Download OpenClaw Docker E2E package
115115
if: inputs.hydrate-artifacts == 'true' && steps.plan.outputs.needs_package == '1'
116-
uses: actions/download-artifact@v8
116+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
117117
with:
118118
name: ${{ inputs.package-artifact-name }}
119119
path: .artifacts/docker-e2e-package

‎.github/actions/setup-node-env/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ runs:
139139
140140
- name: Save pnpm store cache
141141
if: ${{ inputs.install-deps == 'true' && inputs.use-actions-cache == 'true' && inputs.save-actions-cache == 'true' && runner.os != 'Windows' && steps.setup-pnpm.outputs.store-cache-hit != 'true' }}
142-
uses: actions/cache/save@v5
142+
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
143143
with:
144144
path: ${{ steps.setup-pnpm.outputs.store-path }}
145145
key: ${{ steps.setup-pnpm.outputs.store-cache-primary-key }}

‎.github/actions/setup-pnpm-store-cache/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ runs:
9292
- name: Restore pnpm store cache
9393
id: pnpm-store-cache
9494
if: ${{ inputs.use-actions-cache == 'true' && runner.os != 'Windows' }}
95-
uses: actions/cache/restore@v5
95+
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
9696
with:
9797
path: ${{ steps.pnpm-store.outputs.path }}
9898
key: pnpm-store-${{ runner.os }}-${{ runner.arch }}-${{ inputs.node-version }}-${{ hashFiles(inputs.package-manager-file) }}-${{ hashFiles(inputs.lockfile-path) }}

‎.github/workflows/auto-response.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,24 +25,24 @@ jobs:
2525
pull-requests: write
2626
runs-on: ubuntu-24.04
2727
steps:
28-
- uses: actions/checkout@v6
28+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
2929
with:
3030
ref: ${{ github.sha }}
3131
persist-credentials: false
32-
- uses: actions/create-github-app-token@v3
32+
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
3333
id: app-token
3434
continue-on-error: true
3535
with:
3636
app-id: "2729701"
3737
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
38-
- uses: actions/create-github-app-token@v3
38+
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
3939
id: app-token-fallback
4040
if: steps.app-token.outcome == 'failure'
4141
with:
4242
app-id: "2971289"
4343
private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }}
4444
- name: Run Barnacle auto-response
45-
uses: actions/github-script@v9
45+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
4646
with:
4747
github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }}
4848
script: |

‎.github/workflows/ci-build-artifacts-testbox.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@ jobs:
140140
141141
- name: Restore dist build cache
142142
id: dist-cache
143-
uses: actions/cache/restore@v5
143+
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
144144
with:
145145
path: |
146146
.artifacts/build-all-cache/
@@ -175,7 +175,7 @@ jobs:
175175
176176
- name: Save dist build cache
177177
if: steps.dist-cache.outputs.cache-hit != 'true'
178-
uses: actions/cache/save@v5
178+
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
179179
with:
180180
path: |
181181
.artifacts/build-all-cache/

0 commit comments

Comments
 (0)