Repository navigation
Description
Description
kagent's default amd64 gVisor runsc (gs://gvisor/releases/nightly/2026-05-19/x86_64/runsc, set in go/core/pkg/app/app.go) cannot restore node.js OpenClaw/NemoClaw actors on the substrate runtime. The golden snapshot checkpoints fine, but ResumeActor → runsc restore of the openclaw sub-container fails with inconsistent private memory files on restore, so the AgentHarness is stuck ActorReady=False (ActorResuming) indefinitely. The bug reproduces only on the amd64 build of the runsc nightly — arm64 restores fine with identical code.
Root cause
A gVisor checkpoint/restore bug in the amd64 build of the pinned 2026-05-19 runsc nightly. It is fixed in a later gVisor nightly: 2026-06-02 checkpoints AND restores successfully, and is before the 2026-06-13 build that breaks checkpoint outright — so 2026-06-02 is a working window. The defect is in the runsc binary, not the substrate version or the kagent controller (confirmed below).
Steps to reproduce
- substrate
0.0.6+ kagent0.9.8on an amd64 cluster, default runsc (2026-05-19). - Create
AgentHarness {runtime: substrate, backend: openclaw}with a validgatewayTokenSecretRef+modelConfigRef. RunWorkload(boot) OK →CheckpointWorkloadOK →ActorTemplateReady=True.ResumeActor→runsc restoreof theopenclawcontainer fails.
Outcome (actual)
Restore fails on amd64 and the harness never reaches Ready:
FATAL ERROR: starting sub-container [/bin/sh -c ... openclaw gateway run --port 80 --allow-unconfigured]:
inconsistent private memory files on restore:
savedMFOwners = [pause:/], mfmap = map[openclaw:/:unwasteSmall: ...]
ate-api: ResumeActor ... workflow failed at step CallAteletRestore:
while creating workload from golden snapshot: rpc error: code = Internal desc = internal server error
Expected outcome
The OpenClaw/NemoClaw actor restores from its golden snapshot and the AgentHarness reaches Ready / ActorRunning — as it already does on arm64, and as it does on amd64 once runsc is bumped to 2026-06-02 (a fresh harness then reaches Ready / ActorRunning in ~104s).
Evidence / isolation
- Cross-arch: the identical stack (same ateom build, same runsc date, same
nemoclaw/sandbox-base@sha256:d52bee…image, samesystrapplatform) restores on arm64 (Kind, Apple Silicon) and fails on amd64 (Talos). Not the substrate version — rebuildingateom-gvisorfrom substrateHEADon amd64 reproduces the same failure. - amd64 runsc sweep:
2026-05-19❌,2026-05-26❌ (bothinconsistent private memory files),2026-06-02✅ (checkpoint + restore),2026-06-13breaks checkpoint. - Caveat: the arm64 repro ran on a different host kernel (Docker Desktop VM), so an arch-vs-kernel contribution can't be 100% ruled out. The provable, actionable claim independent of that:
2026-05-19amd64 fails to restore,2026-06-02fixes it and still checkpoints.
Suggested fix
Bump the amd64 runsc default to 2026-06-02 (sha256 efd12935f6654c91a1389710eb8dfa4d12b6b9be00db87526dc2eb584ad00119); leave arm64 unchanged. Fix submitted in #2035.
Workaround (helm, amd64 only — arm64 default is fine):
controller:
substrate:
runscAMD64URL: "gs://gvisor/releases/nightly/2026-06-02/x86_64/runsc"
runscAMD64SHA256: "efd12935f6654c91a1389710eb8dfa4d12b6b9be00db87526dc2eb584ad00119"Note: the runsc-compatibility coupling is conceptually owned by
kagent-dev/substrate(its C/R is built against a specific runsc version), but that repo has issues disabled, so filing here alongside the fix.
Environment
- Talos Linux v1.12.2 (Sidero), amd64 · Kubernetes v1.35.0
- kagent
v0.9.8· substrate chart0.0.6(ateom-gvisorv0.0.6) · gVisor platformsystrap
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsBacklog
Activity
fix(substrate): bump amd64 runsc default to gVisor nightly 2026-06-02 (…