Skip to content

Auto-update can leave gateway running with stale hashed chunk imports until restart #86407

Description

@chac4l

Summary

After an automatic package update, the running gateway process can continue using an older in-memory module graph while files under the installed package have already been replaced. Subsequent lazy imports then fail with Cannot find module for hashed bundle chunks that no longer exist on disk.

This breaks channel middleware and cron/agent execution until the gateway process is restarted manually.

Observed symptoms

Examples of sanitized errors from the running process after auto-update:

Cannot find module '/usr/lib/node_modules/openclaw/dist/commands-status-<old-hash>.js' imported from /usr/lib/node_modules/openclaw/dist/get-reply-<old-hash>.js
Cannot find module './run-attempt-<old-hash>.js'
Require stack:
- ~/.openclaw/npm/node_modules/@openclaw/codex/dist/harness.js

The files on disk were already consistent with the updated version and referenced new chunk hashes. Restarting the gateway loaded the new module graph and restored channel handling and Codex cron execution.

Expected behavior

Auto-update should not leave a running gateway in a mixed old-process/new-files state. One of these should happen:

  • auto-update performs a controlled gateway restart after replacing hashed bundles;
  • auto-update preserves old chunk files until all running processes exit;
  • auto-update refuses to replace runtime-loaded bundle files while the gateway is active and reports that restart is required.

Actual behavior

The update completed while the old process remained active. Lazy imports continued to reference removed chunks, so inbound commands and scheduled Codex jobs failed until manual restart.

Impact

  • Inbound channel commands can stop responding.
  • Scheduled jobs can fail with misleading Cannot find module errors.
  • The gateway health endpoint may still look superficially alive or recover after restart, so the real failure mode is easy to misdiagnose as a model/Codex issue.

Suggested fix

Make package auto-update atomic with respect to the running gateway. The safest default is probably: after a successful auto-update of the runtime package or bundled plugins, trigger a controlled restart or set a durable restart_required state and pause channel/cron processing until restart completes.

Version

Observed on 2026.5.22 package install.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions