Skip to content

Need a way to recover an actor from CRASHED state #1556

Closed

Description

Today an Actor that gets into CRASHED state cannot be recovered. We should have a way to "rollback" to the latest saved snapshot and resume. There are different ways of encoding this into the API. A few options I can think of are a) an explicit RollbackActor (or RevertActor) method which flips the state back to SUSPENDED b) allow ResumeActor to just resume (this works today if we remove the precondition check for CRASHED) c) allow SuspendActor to switch from CRASHED to SUSPENDED. Others?

Related to #119

Activity

  1. sfunkenhauser commented on Sep 9, 2026

    @sfunkenhauser
    Collaborator

    Some discussion about this on #292 as well.

  2. zoez7 commented on Sep 9, 2026

    @zoez7
    Collaborator

    a variant of b) ResumeActor can resume with ResumeActorOptions {allow_crashed}?
    This allows users to ack "there could be data loss", without creating a new API.

  3. BenTheElder commented on Sep 9, 2026

    @BenTheElder
    Collaborator

    In the future we might have a "keep the partial disk state" option so probably not just "resume allow_crashed: true/false"

  4. zoez7 commented on Sep 9, 2026

    @zoez7
    Collaborator

    Another option is d) "crash is a terminal state, you can only create another actor from the snapshot to recover" which should be already support today with the ActorSnapshot and Tags API.

  5. laoj2 commented on Sep 9, 2026

    @laoj2
    Collaborator

    #1541 explicitly assumes that CRASHED is a terminal state. We'll need to rework the snapshot logic of crashed actors too if we add a way to recover an actor from CRASHED.

  6. juli4n commented on Sep 10, 2026

    @juli4n
    CollaboratorAuthor

    How about a RevertActor RPC? The idea is that it would reset back a CRASHED actor to SUSPENDED (to the latest external snapshot) and would also suspend a RUNNING actor and revert to the latest saved snapshot (basically suspend but don't checkpoint the state).

  7. zlammerts-svg commented on Sep 17, 2026

    @zlammerts-svg
    Collaborator

    Can I assume we do not need Zoe for this one any more? : )

  8. added a commit that references this issue on Sep 18, 2026
    944abe3
  9. shrutiyam-glitch commented on Sep 30, 2026

    @shrutiyam-glitch
    Collaborator

    Note: As mentioned here - https://github.com/agent-substrate/substrate/blob/ce05e5d93b9a0fe6155aa3af1ff8784fff6954ff/docs/api-guide.md#revertactor - external volumes are reverted since they are not a part of the snapshot.
    This issue is being tracked here - #1951

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area/apiUser-facing API changesarea/reliabilitykind/designDesign discussion, investigation, or research required before implementation

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions