Skip to content

Worker Self-reports Snapshot Compatibility #2348

Closed

Description

@zoez7

1. Proposed API

Important

API Update (from initial issue description):

  1. Moved versioning up from HardwareIdentity to SandboxRuntime (VersionedSandboxCompat): Runtime binary compatibility (hash / warm_boot_hashes) and guest-visible hardware compatibility (HardwareIdentity) face the exact same schema evolution problem, so they are unified into a single versioned structure on SandboxRuntime.
  2. Folded string hash into repeated AttributeEntry attributes: Both runtime asset digests (e.g. gvisor_sha256, pause_image) and effective guest hardware features (e.g. architecture, cpu_vendor, cpu_model) are represented as AttributeEntry key-value pairs under VersionedSandboxCompat, matched by exact equality within the same version.
Previous draft (superseded — click to expand)
message SandboxRuntime {
  // e.g. "gvisor", "microvm"
  // +k8s:required
  string sandbox_class = 1;

  // Primary compatibility key, computed by taking a hash of all runtime assets
  // +k8s:required
  string hash = 2;

  // In the future, we can add additional older runtime hash that this runtime can warm-boot / restore.
  // we don't have to add this for GA yet.
  // repeated string warm_boot_hashes = 3; 

  // SandboxConfig name, informational only, never used during scheduling.
  // +k8s:optional
  string name = 4;

  // Effective hardware features the guest sees (after the mask)
  // +k8s:required
  HardwareIdentity hardware_identity = 5;
}

message HardwareIdentity {
  // This is the version that gets included in an actor's snapshot once a sprint finishes on this worker.
  VersionedHardwareIdentity version = 1;

  // These are the versions supported by this worker, during scheduling, a snapshot
  // will warm boot as long as it matches one of the compatible_versions.
  // [not in GA, can add later]
  // repeated VersionedHardwareIdentity compatible_versions = 2;
}

message VersionedHardwareIdentity {
  // +k8s:required
  string version = 1;

  // Key-value hardware attributes defined by this schema version.
  // Matched by exact equality within the same version.
  // +k8s:required
  repeated AttributeEntry attributes = 2;
}

service WorkerService {
  rpc Register(RegisterRequest) returns (RegisterResponse);
}

message RegisterRequest {
  // +k8s:required
  ObjectRef worker = 1;

  // +k8s:required
  WorkerResources capacity = 2;

  // Runtime that new and cold-booted actors start under.
  // Snapshots matching default_runtime can also warm-boot on this worker.
  // +k8s:required
  SandboxRuntime default_runtime = 3;

  // Additional runtimes (excluding default_runtime) that this worker can
  // restore snapshots from right now.
  // +k8s:optional
  // +k8s:maxItems=32
  repeated SandboxRuntime restorable_runtimes = 4;
}

SandboxRuntime API:

message SandboxRuntime {
  // e.g. "gvisor", "microvm"
  // +k8s:required
  string sandbox_class = 1;

  // SandboxConfig or version name (e.g. "gvisor-2"), informational only,
  // never used during scheduling.
  // +k8s:optional
  string name = 2;

  // Primary compatibility version stamped onto an actor's snapshot when a
  // checkpoint is taken on this runtime. Snapshots matching `version` can
  // warm-boot on this runtime.
  // +k8s:required
  VersionedSandboxCompat version = 3;

  // [Post-GA] Additional compatibility versions (excluding `version`) that can
  // warm-boot directly onto this runtime (either an older schema version like
  // "v1", or an older runtime asset digest whose checkpoints this binary can
  // restore).
  // During scheduling, a snapshot can warm-boot on this runtime if its version
  // matches either `version` or one of `compatible_versions`.
  // When the actor suspends again, its new snapshot is stamped with the primary `version`.
  // +k8s:optional
  // repeated VersionedSandboxCompat compatible_versions = 4;
}

message VersionedSandboxCompat {
  // Compatibility schema version (e.g. "v1") defining which runtime and
  // hardware attributes are recorded and how they are computed.
  // +k8s:required
  string version = 1;

  // Key-value compatibility attributes defined by this schema version, covering
  // both runtime assets (e.g. "gvisor_asset_hash") and effective guest hardware
  // features after masking (e.g. "architecture", "cpu_features").
  // Matched by exact equality of all keys and values within the same version.
  // +k8s:required
  repeated AttributeEntry attributes = 2;
}

message AttributeEntry {
  string key = 1;
  string value = 2;
}

Snapshot & atelet Wire Contract

  • On Snapshot (and ActorTemplateStatus.golden_snapshots): Each memory snapshot records the SandboxRuntime it was taken on (sandbox_class, informational name, and the primary version).
  • On ateapi $\rightarrow$ atelet (RunRequest and RestoreRequest):
    • do not send SandboxAssets in requests to atelet.
    • RunRequest carries the worker's default_runtime (SandboxRuntime).
    • RestoreRequest carries the matched SandboxRuntime from the worker's restorable_runtimes, if the snapshot matches, if not, it carries the worker's default_runtime (SandboxRuntime), and cold boots

Activity

  1. added a commit that references this issue on Oct 9, 2026
    2eda16a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area/apiUser-facing API changesarea/nodekind/featureAn enhancement / feature request or implementation

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions