Skip to content

fix: recover live Gateway snapshot updates without a control build#104964

Merged
steipete merged 1 commit into
mainfrom
codex/live-updater-precontrol-build
Jul 12, 2026
Merged

fix: recover live Gateway snapshot updates without a control build#104964
steipete merged 1 commit into
mainfrom
codex/live-updater-precontrol-build

Conversation

@steipete

@steipete steipete commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Closes #104945

What Problem This Solves

Fixes an issue where operators updating a managed macOS Gateway from an immutable runtime snapshot would remain blocked when a concurrent source update invalidated the checkout's last exact control build while the snapshot service was still running.

Why This Change Was Made

When native inspection proves the snapshot is still live, the updater now builds the verified clean source checkout solely to obtain an exact trusted suspension client, then uses that client to fence work before bootout and retargeting. This recovery is explicitly forbidden for a LaunchAgent that targets the source checkout, and snapshot code remains unexecuted.

User Impact

Snapshot-backed managed Gateways can recover automatically from missing or stale source control builds without weakening the active-work fence or mutating a live source-backed dist tree. Already-stopped snapshots retain the existing native recovery path. No release or version change is included.

Evidence

  • Live clawmac reproduction on landed main: updater failed closed with Gateway control is unavailable and the managed Gateway could not be proven stopped while gateway-508f6a2 owned port 18789.
  • node scripts/run-vitest.mjs test/scripts/openclaw-live-updater.test.ts — 44 passed.
  • Targeted oxfmt and git diff --check — clean.
  • Fresh autoreview after adding the snapshot-only guard — no accepted or actionable findings.
  • Two Blacksmith Testbox attempts were shut down during provider hydration before any command executed; exact-head CI and the mandatory hosted landing gate remain required before merge.

Live clawmac migration will be retried with the exact landed follow-up implementation. No release.

@openclaw-barnacle openclaw-barnacle Bot added size: S maintainer Maintainer-authored PR labels Jul 12, 2026
@steipete
steipete force-pushed the codex/live-updater-precontrol-build branch from c4f0fde to 564eb5f Compare July 12, 2026 04:37
@clawsweeper clawsweeper Bot mentioned this pull request Jul 12, 2026
Closed
@steipete steipete mentioned this pull request Jul 12, 2026
Merged
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels Jul 12, 2026
@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 12, 2026, 12:46 AM ET / 04:46 UTC.

Summary
The PR adds a snapshot-only recovery path that builds an exact trusted source Gateway client before suspension when a running immutable snapshot lacks a usable control build, with focused regression coverage and updater guidance.

PR surface: Tests +87, Docs 0, Other +41. Total +128 across 3 files.

Reproducibility: yes. Live current-main validation observed the exact fail-closed error while the detached snapshot still owned port 18789, and source inspection confirms the matching branch.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #104945
Summary: This PR is the direct candidate fix for the canonical live-reproduced snapshot updater issue, following a merged predecessor that implemented most of the snapshot migration but missed this active-snapshot recovery state.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh the mandatory exact-head landing gate after the unrelated plugin boundary failure is resolved, then perform the documented post-land live clawmac migration.

Risk before merge

  • [P1] The exact-head build-artifacts check currently fails in an untouched current-base plugin boundary test, so the mandatory landing gate must be refreshed or repaired before merge even though this updater patch did not cause that failure.
  • [P1] The after-fix live clawmac migration is intentionally deferred until the exact implementation lands, leaving post-land operational proof of suspension, retargeting, and listener ownership outstanding.

Maintainer options:

  1. Decide the mitigation before merge
    Land the guarded snapshot-only source-control build path after the exact-head gate is clean, then run the documented real managed-snapshot migration and require successful suspension, native bootout, source retargeting, restart health, log audit, and exact listener ownership before closing the linked issue.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] No automated code repair is indicated; maintainers should clear the unrelated exact-head landing failure, land through the protected workflow, and complete the planned post-land live migration proof.

Security
Cleared: The diff adds no dependency source, permission, secret, or third-party execution surface and runs the existing frozen install/build commands only after strict canonical-checkout and snapshot-provenance validation.

Review details

Best possible solution:

Land the guarded snapshot-only source-control build path after the exact-head gate is clean, then run the documented real managed-snapshot migration and require successful suspension, native bootout, source retargeting, restart health, log audit, and exact listener ownership before closing the linked issue.

Do we have a high-confidence way to reproduce the issue?

Yes. Live current-main validation observed the exact fail-closed error while the detached snapshot still owned port 18789, and source inspection confirms the matching branch.

Is this the best way to solve the issue?

Yes. Building only the verified canonical source checkout to obtain the existing suspension client for an isolated snapshot is narrower and safer than executing snapshot code, weakening stopped proof, or introducing another control mechanism.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4139938dcf18.

Label changes

Label changes:

  • add P2: The PR repairs a verified stable managed macOS Gateway update failure with a bounded snapshot-deployment blast radius.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply because this PR is explicitly maintainer-labeled; its body nevertheless records the exact before-fix live macOS failure and the planned post-land migration proof.

Label justifications:

  • P2: The PR repairs a verified stable managed macOS Gateway update failure with a bounded snapshot-deployment blast radius.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply because this PR is explicitly maintainer-labeled; its body nevertheless records the exact before-fix live macOS failure and the planned post-land migration proof.
Evidence reviewed

PR surface:

Tests +87, Docs 0, Other +41. Total +128 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 1 87 0 +87
Docs 1 1 1 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 53 12 +41
Total 3 141 13 +128

What I checked:

Likely related people:

  • steipete: Introduced the merged snapshot migration, performed the exact live follow-up validation, and authored this recovery across the central updater, test, and workflow-guidance files. (role: snapshot updater feature owner and follow-up implementer; confidence: high; commits: f802a6f8bc09, 564eb5f964cb; files: .agents/skills/openclaw-live-updater/SKILL.md, .agents/skills/openclaw-live-updater/scripts/update-main.mjs, test/scripts/openclaw-live-updater.test.ts)
  • Vincent Koc: Commit c37c8d5 introduced the live-updater skill, implementation module, and initial test module that the later snapshot work extends. (role: original updater framework contributor; confidence: medium; commits: c37c8d567139; files: .agents/skills/openclaw-live-updater/SKILL.md, .agents/skills/openclaw-live-updater/scripts/update-main.mjs, test/scripts/openclaw-live-updater.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete force-pushed the codex/live-updater-precontrol-build branch from 564eb5f to 05aaec9 Compare July 12, 2026 04:55
@steipete

steipete commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Land-ready proof for exact head 05aaec9de62b497f8cae49fe98d616d6b75e2860:

  • Full hosted CI run 29180421704: 50 successful jobs, no failures or pending jobs.
  • Focused updater regression proof: node scripts/run-vitest.mjs test/scripts/openclaw-live-updater.test.ts — 44/44 passed.
  • Targeted oxfmt and git diff --check passed.
  • Fresh autoreview after adding the snapshot-only hard gate reported no accepted or actionable findings.
  • Review artifacts validated with zero findings. OPENCLAW_TESTBOX=1 scripts/pr prepare-run 104964 reused exact-head hosted proof successfully.
  • Live clawmac reproduced the guarded state: launchd/listener PID 2982 continues serving the validated immutable snapshot while the updater fails closed. This PR supplies the exact trusted source control-build recovery path without executing snapshot code or allowing a live source-tree build.

After merge I will rerun the updater against that live state and require exact source retarget, launchd/listener ownership, deep RPC health, restart-window logs, Mac app attach-only, node fleet, and agent task proof before closing #104945.

@steipete
steipete merged commit 2263272 into main Jul 12, 2026
77 checks passed
@steipete
steipete deleted the codex/live-updater-precontrol-build branch July 12, 2026 05:02
@steipete

steipete commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Merged via squash.

@github-actions github-actions Bot mentioned this pull request Jul 12, 2026
Open
@clawsweeper clawsweeper Bot mentioned this pull request Jul 12, 2026
Closed
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 12, 2026
@steipete
fix(updater): build snapshot control client safely (openclaw#104964)
050cba4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Updater can mis-handle managed Gateway runtime snapshots

1 participant

@steipete