Add a sandbox quarantine command with durable restart fencing and receipts #10140

Problem Statement

NemoClaw has no operator command that stops a suspicious sandbox, blocks automatic or manual restart, and preserves its state for inspection.

nemoclaw <name> stop stops the runtime provider workload but preserves the sandbox registry, workspace, credentials, and shared gateway state. A later start can resume it. nemoclaw <name> destroy removes the sandbox and its registry entry. Neither operation provides a durable emergency-control state with a secret-free receipt.

Operators need this boundary when an agent behaves unexpectedly, consumes resources without authorization, or may have attempted unauthorized access. The operation must stop execution and sandbox access without deleting workspace state or evidence.

Desired Behavior

Add a user-facing sandbox quarantine operation. Proposed syntax:

nemoclaw <name> quarantine --reason "<reason>"

The final command name requires maintainer approval because NemoClaw already uses quarantine for other blocked states.

For one registered sandbox, the operation must:

1. Acquire the existing sandbox lifecycle lock.
2. Resolve the runtime provider and exact target identity from the sandbox registry.
3. Reject missing, ambiguous, stale, or replaced target identity before mutation.
4. Record a durable quarantine fence and operation journal before ordinary lifecycle commands can reactivate the sandbox.
5. Stop messaging channels, dashboard and service access paths, and the runtime provider workload through their owning interfaces.
6. Verify that execution and sandbox access stopped.
7. Preserve the sandbox registry entry, workspace, snapshots, logs, and other evidence.
8. Write a secret-free receipt with the request identity, target identity, lifecycle generation, attempted operations, observed outcomes, timestamps, and failures.

start, recover, rebuild, onboarding reuse, restore, and upgrade paths must refuse to reactivate a quarantined sandbox. Read-only status and evidence collection should remain available when they do not weaken the fence. destroy may remain available through its existing confirmation and exact-target cleanup contract.

A separate explicit operation must release the quarantine fence. Releasing the fence must not start the sandbox. The operator must run start separately.

Repeated requests with the same idempotency key must reconcile the existing operation and return its receipt. A transport timeout must trigger target-state inspection. It must not trigger an automatic mutation through another transport.

Example:

$ nemoclaw my-agent quarantine --reason "unexpected outbound activity"
Sandbox 'my-agent' is quarantined.
Execution: stopped and verified
Sandbox access: stopped and verified
Workspace: preserved
Receipt: <path>

Proposal Details

Scope and Exclusions

In scope:

• One sandbox-scoped emergency-control operation.
• A durable restart fence and crash-recoverable operation journal.
• Exact runtime-provider and target identity checks.
• Idempotent reconciliation after interruption or ambiguous transport results.
• Secret-free receipts that separate enforcement results from observed postconditions.
• Guards on every supported path that can reactivate or replace the sandbox.
• User documentation and focused runtime evidence.

Out of scope:

• A fleet, tenant, or remote control service.
• A second sandbox registry or lifecycle authority.
• Canonical workflow run_id storage. A future run-level adapter may invoke the sandbox operation for each resolved target.
• Direct Docker or Podman commands that bypass the selected runtime provider or OpenShell.
• Automatic revocation of credentials at external providers.
• Automatic deletion of the sandbox, workspace, or evidence.
• A claim of atomic multi-target quarantine.

The maintainer decision must define behavior when the exact target is proven but NemoClaw cannot persist the durable fence. The implementation must not report quarantine success unless both enforcement and required observations complete.

Ongoing Ownership

Proposed owner: NemoClaw runtime integration maintainers. An accountable maintainer must be named before implementation starts.

Placement and Support Expectations

Requested placement: core NemoClaw.

This proposal adds a user-visible sandbox lifecycle and security control. Maintainers must record Accept, Request changes, Defer, or Decline. An Accept decision must name the accountable maintainer and the required validation evidence.

Validation Plan

• Unit tests for command input, reason redaction and bounds, target resolution, idempotency, receipt shape, and every lifecycle guard.
• Negative tests for missing registry state, stale lifecycle generation, target replacement, wrong gateway, unsupported runtime provider, concurrent mutation, and unauthorized release.
• Failure tests for partial channel or service shutdown, runtime-provider failure, transport ambiguity, observation failure, receipt persistence failure, interruption, retry, and cleanup.
• Tests that start, recover, rebuild, onboarding reuse, restore, and upgrade cannot bypass an active fence.
• Tests that status, doctor, and approved evidence collection do not remove or weaken the fence.
• Tests that receipts and diagnostics contain no credential values, policy bodies, message content, or user task data.
• A live E2E test using a real OpenShell sandbox. It must verify execution stops, access paths stop, state remains available, restart is denied, repeated invocation is idempotent, and explicit release does not start the sandbox.
• Runtime-provider evidence for each provider accepted by the maintainer decision. An unsupported provider must fail before mutation.

Compatibility Requirements

The operation must extend the current runtime-provider bundle and sandbox lifecycle lock. It must not add a parallel provider registry.

Availability is capability-based. A runtime provider without the required stop and observation behavior must return a nonzero result before mutation. The issue does not require a direct container-engine fallback.

Related work:

• Isolate the OpenShell integration and migrate eligible CLI operations to the SDK or gRPC #9802 owns the typed NemoClaw OpenShell interface and transport migration. This issue must use that interface when the required slice exists and must not add an automatic cross-transport mutation fallback.
• Epic: unify sandbox mutation lock decisions across sync and async callers #9955 owns shared lifecycle-lock decisions. This issue must reuse the accepted lock decision model and must not create another lock implementation.

No open issue or PR found in the duplicate search provides this user-facing operation.

Security or Privacy Impact

This command crosses the host, NemoClaw state, OpenShell gateway, sandbox, policy, credential, inference, and messaging trust boundaries.

Required controls:

• Authenticate and authorize the exact sandbox and action at the owning boundary.
• Bind mutations to the recorded gateway, runtime provider, lifecycle generation, and live target identity.
• Persist the fence before an ordinary lifecycle path can reactivate the target.
• Fail closed on ambiguous, replaced, or unproven target identity.
• Keep the fence active after partial enforcement, interruption, or inconclusive verification.
• Preserve credential custody. Do not copy credential values into the registry, receipt, logs, arguments, or sandbox.
• Treat runtime-provider responses as enforcement claims. Record independent observed postconditions separately.
• Require explicit release. Release must verify the same fence and target generation and must not start the sandbox.

The receipt may contain sandbox and provider identifiers, reason text, timestamps, operation outcomes, and evidence references. It must not contain credentials, policy bodies, prompts, tool inputs, tool outputs, messages, or workspace content.

Implementation Idea

Extend the existing runtime-provider bundle with one bounded quarantine capability. Reuse SandboxEntry.lifecycleGeneration, lifecycleLiveIdentityFingerprint, gatewayName, and gatewayPort for exact target binding.

Implement the orchestration beside the current sandbox stop action. Reuse the sandbox lifecycle lock and provider-specific lifecycle surface. Add durable fence and operation-receipt state to the existing NemoClaw sandbox registry or its established transaction state, subject to the accepted design.

The first capability slice should quarantine one registered sandbox through its selected runtime provider. Run-level fanout and any missing OpenShell enforcement primitives should remain separate follow-up work.

Category

Feature

Checklist

• I searched existing issues and this is not a duplicate
• I described the problem and desired behavior