Skip to content

msb adapter: re-run kit startup commands on msb start/restart (durable across stop/reboot) #247

Closed

Description

@mogul

Summary

On the msb backend, kit startup commands do not re-run on a raw msb start / msb restart (or a host-reboot-driven guest restart). They re-run only when acq itself re-drives an apply (i.e. acq run <sandbox> → acq_backend_ensure_kits_applied). This means a sandbox that is stopped and started back up outside acq comes back without its kit-defined background services running.

This is the durability half of the problem that #239 explicitly parked — #239 is scoped as an adapter-internal hardening refactor ("stages commands at create time", "no change to observable behavior"), whereas this issue is a behavior-level lifecycle gap. Filing separately so the two don't get conflated; revisiting #239 is expected as part of the fix.

Why this matters (concrete failure)

The openchamber acq mixin kit (in agentic-coding-patterns, integrations/isolation/acq-kits/openchamber/) runs a long-lived, supervised stack — a shared opencode serve on :4096 and OpenChamber on :3000 — via a commands: [{phase: startup, background: true}] entry that launches openchamber-start.sh.

On msb this works while the sandbox keeps running (msb applies no idle_timeout, so the VM never idles out — confirmed in microsandbox crates/runtime/lib/heartbeat.rs: check(None) never reclaims). But after msb stop + msb start, or a host reboot followed by a raw msb start, the VM is back but openchamber-start.sh never re-runs, so the browser UI is dark until the user goes back through acq run.

Root cause (traced)

  • microsandbox would re-run a persisted LaunchConfig.startup on Sandbox::start_detached() (used by both msb start and msb restart — see microsandbox crates/cli/lib/commands/start.rs, restart.rs). The mechanism exists guest-side.
  • acq never populates LaunchConfig.startup. acq.backends/msb.sh applies kit startup commands imperatively via msb exec at provision time and on the acq run re-attach heal loop (acq_backend_ensure_kits_applied, msb.sh:2054-2078). The only msb create invocation (msb.sh:1204) carries no startup/script flag.
  • acq has no start/restart/resume verb, so it never re-drives apply on a bare microsandbox restart.
  • The background: true nohup detach (msb.sh:848-869) and create-time -p ports are provision-time constructs; the detached process does not survive a restart.

There is also a design-vs-implementation gap: docs/explorations/acq-design.md:196 states msb runs startup "after every msb start", but the shipped adapter achieves "after every acq re-apply", not after a raw msb start.

Proposed direction (to be designed; revisits #239)

Make msb kit startup (and initFiles) re-run on a microsandbox-native restart, without breaking the mid-life apply path. Candidate approaches to evaluate:

  1. Persist startup via microsandbox script registration — stage the kit's startup as a --script/--script-path at msb create so microsandbox re-runs it on every start_detached. This is exactly the create-time-only, script-file-bodied case that the msb adapter: use --script for kit-command injection instead of copy+sh -c #239 design note (msb.sh:406-412) called out as the one clean win for --script but left parked. Keep the existing msb exec path for the mid-life apply/heal case (the msb adapter: use --script for kit-command injection instead of copy+sh -c #239 timing objection); the two are complementary, not mutually exclusive.
  2. Give acq a start/restart verb that re-drives acq_backend_ensure_kits_applied so recovery through acq is a first-class, documented path.
  3. Some combination — native persistence for the reboot/msb start path, acq re-apply for the in-place kit-add path.

Whichever lands needs an ADR (touches the #239 decision + ADR-0011/ADR-0014 command/lifecycle vocabulary) and must keep the neutral hybrid/v1 kit vocabulary and idempotency/marker semantics intact.

Acceptance criteria

  • After msb stop <sandbox> + msb start <sandbox> (no acq re-invocation), kit startup commands run again and background supervisors come back up.
  • The openchamber kit's UI is reachable after a stop/start (and after a reboot + msb start) without an acq run re-attach.
  • Mid-life kit apply/heal (acq run re-attach, acq kit apply) still works and stays idempotent (install-phase marker-gating preserved).
  • docs/explorations/acq-design.md:196 and the msb.sh msb adapter: use --script for kit-command injection instead of copy+sh -c #239 design note reconciled with the shipped behavior.
  • Regression coverage in scripts/test-acq; live verification on a KVM host via scripts/verify-backends.

Related

Key references

  • acq.backends/msb.sh:1204 — the only msb create (no startup/script flag)
  • acq.backends/msb.sh:2054-2078 — acq_backend_ensure_kits_applied (acq-driven re-apply on re-attach)
  • acq.backends/msb.sh:349-412 — msb adapter: use --script for kit-command injection instead of copy+sh -c #239 design note (why exec-based; the parked --script win)
  • acq.backends/msb.sh:848-869 — background: true nohup detach (provision-time only)
  • docs/explorations/acq-design.md:196 — design intent ("after every msb start") not matched by implementation

Note: no idle_timeout-related work is needed — msb leaves it unset, so a running sandbox already keeps startup services alive indefinitely. This issue is strictly about the stop→start / reboot restart path.

AI-assisted (OpenCode); requires human review.

Activity

  1. mogul commented on Aug 6, 2026

    @mogul
    ContributorAuthor

    Resolved by #260 (merged 2026-08-04, commit 7bbc999). Verified against main:

    • acq start / acq restart verbs exist (acq:676, acq:700) and re-drive the idempotent kit apply, which re-runs the startup phase.
    • acq run <stopped-sandbox> works end-to-end — the msb heal starts a stopped sandbox before healing/attaching (_acq_msb_is_running + acq_backend_start, acq.backends/msb.sh), blocking on exec-readiness so the heal never races the async boot.
    • Startup replay is delivered by acq re-driving the apply on acq run/acq start/acq restart; the create-time --script-path acq-startup staging from feat(msb): stage kit startup commands as a create-time script #259 is retained (_acq_msb_stage_startup_script).
    • Secret re-injection on resume — msb start re-reads the sandbox's --secret ENV@HOST bindings via the shared _acq_msb_bind_secrets_into (values move via transient env vars, never argv).
    • Design docs reconciled — docs/explorations/acq-design.md and ADR-0017 now state that resume is via acq start/restart/run and that a raw msb start outside acq is not a supported resume path (it can't boot a secret-bound sandbox).

    Live verification (KVM host, per #260): scripts/verify-backends 0 failures; stop→start restored a background-startup kit's supervisor; acq run <stopped-sandbox> reattached cleanly.

    Note on the parked half: the general "move ALL kit-command injection off sh -c" refactor (#239) was intentionally kept exec-based for the install + mid-life-apply phases (see the design note in acq.backends/msb.sh); only the create-time startup slice moved to --script-path. That distinction is tracked in #239.

    Closing as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions