Skip to content

Fix restart continuations after explicit restarts#73812

Closed
VACInc wants to merge 2 commits into
openclaw:mainfrom
VACInc:fix-restart-continuation-topic-replies
Closed

Fix restart continuations after explicit restarts#73812
VACInc wants to merge 2 commits into
openclaw:mainfrom
VACInc:fix-restart-continuation-topic-replies

Conversation

@VACInc

@VACInc VACInc commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Force restart-sentinel continuation turns to use automatic source delivery so owed restart replies remain visible in group/topic conversations.
  • Bound explicit chat/tool restart deferral so a leaked or long-lived pending counter cannot make gateway.restart return ok without ever emitting SIGUSR1.
  • Add regression coverage for Telegram-style group topic restart continuations and per-request restart deferral timeouts.

Root cause

Recent group reply handling defaults normal group/channel turns to message-tool-only delivery. Restart-sentinel continuations use the generic gateway dispatch path, so their final reply could be suppressed in a Telegram topic even though the continuation turn ran.

A second issue showed up in topic 22684: the gateway.restart tool wrote a continuation and returned ok, but the scheduler waited indefinitely in the pre-restart deferral layer. With no runtime deferral timeout, any stale pending counter could prevent SIGUSR1 from ever being emitted.

Validation

  • pnpm exec oxfmt --check --threads=1 src/infra/restart.ts src/infra/infra-runtime.test.ts src/agents/tools/gateway-tool.ts src/agents/tools/gateway-tool.test.ts src/auto-reply/reply/commands-session.ts src/gateway/server-restart-sentinel.ts src/gateway/server-restart-sentinel.test.ts
  • pnpm test src/infra/infra-runtime.test.ts src/agents/tools/gateway-tool.test.ts src/gateway/server-restart-sentinel.test.ts

@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S labels Apr 28, 2026
@clawsweeper

clawsweeper Bot commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge.

Summary
The PR adds automatic source reply delivery for restart-sentinel continuations, passes a 30-second deferral timeout from explicit restart entry points, and extends restart tests.

Reproducibility: yes. The source-level reproduction is to set gateway.reload.deferralTimeoutMs to 0, schedule a no-timeout restart that waits on a nonzero pre-restart pending count, then coalesce an explicit restart with deferralTimeoutMs and observe that the old pending/preparing wait still controls emission.

Next step before merge
A focused automated repair can address the scheduler state gap, shared constant, regression coverage, and changelog entry without product judgment.

Security
Cleared: The diff touches owner-gated restart scheduling and reply-delivery behavior, with no new dependencies, workflow changes, package metadata, secret handling, downloaded artifacts, or broader permissions.

Review findings

  • [P1] Propagate deferral timeouts through coalesced restarts — src/infra/restart.ts:739-746
  • [P3] Share the explicit restart timeout constant — src/agents/tools/gateway-tool.ts:24-25
  • [P3] Add the required changelog entry — CHANGELOG.md:33-35
Review details

Best possible solution:

Update the scheduler so coalesced restarts can update effective timeout state, keep the automatic-delivery override, share one explicit timeout constant, add focused coalescing coverage, and add the required changelog entry.

Do we have a high-confidence way to reproduce the issue?

Yes. The source-level reproduction is to set gateway.reload.deferralTimeoutMs to 0, schedule a no-timeout restart that waits on a nonzero pre-restart pending count, then coalesce an explicit restart with deferralTimeoutMs and observe that the old pending/preparing wait still controls emission.

Is this the best way to solve the issue?

No as submitted. The automatic source-delivery override is the right narrow fix, but the timeout policy must live in pending/preparing restart state rather than only in the first scheduled call's captured opts.

Full review comments:

  • [P1] Propagate deferral timeouts through coalesced restarts — src/infra/restart.ts:739-746
    The new timeout is read from the opts captured by the first scheduled restart. If a later explicit restart with deferralTimeoutMs coalesces onto an already-pending or preparing restart that had no timeout, the pending timer or active deferral poll can still wait indefinitely when runtime config disables the timeout.
    Confidence: 0.91
  • [P3] Share the explicit restart timeout constant — src/agents/tools/gateway-tool.ts:24-25
    The branch defines the same 30_000 timeout in both explicit restart call paths. Export one named constant from the restart module and import it in the gateway tool and chat command paths so the two guarantees cannot drift.
    Confidence: 0.82
  • [P3] Add the required changelog entry — CHANGELOG.md:33-35
    This is a user-facing fix for restart continuation delivery and explicit restart behavior, but the branch does not update CHANGELOG.md. Add a single-line entry under the active Fixes section with appropriate contributor credit.
    Confidence: 0.84

Overall correctness: patch is incorrect
Overall confidence: 0.91

Acceptance criteria:

  • pnpm exec oxfmt --check --threads=1 src/infra/restart.ts src/infra/infra-runtime.test.ts src/agents/tools/gateway-tool.ts src/agents/tools/gateway-tool.test.ts src/auto-reply/reply/commands-session.ts src/auto-reply/reply/commands-session-restart.test.ts src/gateway/server-restart-sentinel.ts src/gateway/server-restart-sentinel.test.ts CHANGELOG.md
  • pnpm test src/infra/infra-runtime.test.ts src/agents/tools/gateway-tool.test.ts src/auto-reply/reply/commands-session-restart.test.ts src/gateway/server-restart-sentinel.test.ts
  • pnpm check:changed in Testbox before handoff if runtime/test files remain changed

What I checked:

  • Current main lacks restart-continuation delivery override: The restart-continuation dispatch call does not pass replyOptions.sourceReplyDeliveryMode, so continuation turns inherit the normal source-reply policy. (src/gateway/server-restart-sentinel.ts:265, 3980eaa1c22e)
  • Group and channel replies default to message-tool-only: resolveSourceReplyDeliveryMode defaults group/channel chat types to message_tool_only unless an explicit requested mode or config selects automatic delivery. (src/auto-reply/reply/source-reply-delivery-mode.ts:28, 3980eaa1c22e)
  • Current main uses only config-level restart deferral timeout: scheduleGatewaySigusr1Restart has no per-request deferralTimeoutMs option and passes cfg.gateway.reload.deferralTimeoutMs into deferGatewayRestartUntilIdle when the scheduled timer fires. (src/infra/restart.ts:777, 3980eaa1c22e)
  • PR timeout state is not propagated on coalescing: The provided PR diff adds deferralTimeoutMs and reads it from the scheduled call's opts, while the existing coalesced branch still only updates emit hooks; a later explicit restart can therefore leave the old timeout state in force. (src/infra/restart.ts:739, ba9a6ec64c8c)
  • Duplicate explicit restart timeout constants: The PR defines EXPLICIT_RESTART_DEFERRAL_TIMEOUT_MS separately in the gateway tool path and the chat /restart path, which can drift across the two explicit restart surfaces. (src/agents/tools/gateway-tool.ts:24, ba9a6ec64c8c)
  • No active changelog entry for this user-facing fix: The active 2026.5.2 Fixes section exists, but this PR does not add an entry for restart continuation visibility or explicit restart timeout behavior. (CHANGELOG.md:33, 3980eaa1c22e)

Likely related people:

  • fuller-stack-dev: Authored the merged fix(gateway): durably hand off restart continuations #70780 durable restart-continuation handoff work that current restart-sentinel continuation behavior builds on. (role: introduced adjacent behavior; confidence: medium; commits: f7550b13366e; files: src/gateway/server-restart-sentinel.ts, src/infra/session-delivery-queue.storage.test.ts, src/infra/session-delivery-queue.recovery.test.ts)
  • steipete: The current local main history/blame points the central restart and gateway continuation files at commit 051171b while preserving the current implementation shape under review. (role: recent maintainer; confidence: low; commits: 051171ba8f2d; files: src/infra/restart.ts, src/gateway/server-restart-sentinel.ts, src/agents/tools/gateway-tool.ts)

Remaining risk / open question:

Codex review notes: model gpt-5.5, reasoning high; reviewed against 3980eaa1c22e.

@openclaw-barnacle openclaw-barnacle Bot added the agents Agent runtime and tooling label Apr 28, 2026
@VACInc VACInc changed the title Fix restart continuations in group topics Fix restart continuations after explicit restarts Apr 28, 2026
@VACInc
VACInc marked this pull request as ready for review April 28, 2026 23:20
@greptile-apps

greptile-apps Bot commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes two bugs: restart-sentinel continuations now force sourceReplyDeliveryMode: "automatic" so owed replies surface in Telegram group/topic threads, and explicit restart callers now pass deferralTimeoutMs: 30_000 to bound an indefinite pre-restart deferral wait. Regression tests cover both the group-topic visibility case and the per-request timeout.

  • The deferralTimeoutMs fix has a gap: if the new explicit restart request is coalesced onto an already-pending restart that lacks a deferral timeout, the coalesced call's opts is never propagated into the pending setTimeout closure, so maxWaitMs stays undefined and the indefinite-wait bug survives in that ordering.
  • EXPLICIT_RESTART_DEFERRAL_TIMEOUT_MS is independently defined in two files rather than being shared from restart.ts.

Confidence Score: 3/5

The sourceReplyDeliveryMode fix is safe, but the deferral-timeout fix has a logical gap in the coalesced restart path that can recreate the original bug.

One P1 finding: deferralTimeoutMs is silently dropped when a new explicit restart is coalesced onto an existing pending restart that lacks a timeout. In the coalesced branch, only emitHooks are updated; the opts captured in the pending setTimeout closure still carries no deferralTimeoutMs, leaving maxWaitMs = undefined and the indefinite-wait bug intact for that code path.

src/infra/restart.ts — specifically the coalesced-restart branch starting at line 694

Comments Outside Diff (1)

  1. src/infra/restart.ts, line 694-711 (link)

    P1 deferralTimeoutMs silently dropped in coalesced path

    When an explicit restart (with deferralTimeoutMs: 30_000) is coalesced into an already-pending restart that was scheduled without a deferral timeout (e.g. an automatic background reload), the else branch updates emitHooks via updatePendingRestartEmitHooks but never updates the timeout. The pending setTimeout closure still references the original opts with no deferralTimeoutMs, so maxWaitMs resolves to undefined and the deferral poll runs indefinitely — the same bug this PR is fixing.

    To close the gap, consider storing deferralTimeoutMs as module-level state (alongside pendingRestartEmitHooks) and reading it from that state inside the setTimeout callback, rather than from a captured closure, so a coalesced call can override it.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/infra/restart.ts
    Line: 694-711
    
    Comment:
    **`deferralTimeoutMs` silently dropped in coalesced path**
    
    When an explicit restart (with `deferralTimeoutMs: 30_000`) is coalesced into an already-pending restart that was scheduled *without* a deferral timeout (e.g. an automatic background reload), the `else` branch updates `emitHooks` via `updatePendingRestartEmitHooks` but never updates the timeout. The pending `setTimeout` closure still references the original `opts` with no `deferralTimeoutMs`, so `maxWaitMs` resolves to `undefined` and the deferral poll runs indefinitely — the same bug this PR is fixing.
    
    To close the gap, consider storing `deferralTimeoutMs` as module-level state (alongside `pendingRestartEmitHooks`) and reading it from that state inside the `setTimeout` callback, rather than from a captured closure, so a coalesced call can override it.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/infra/restart.ts
Line: 694-711

Comment:
**`deferralTimeoutMs` silently dropped in coalesced path**

When an explicit restart (with `deferralTimeoutMs: 30_000`) is coalesced into an already-pending restart that was scheduled *without* a deferral timeout (e.g. an automatic background reload), the `else` branch updates `emitHooks` via `updatePendingRestartEmitHooks` but never updates the timeout. The pending `setTimeout` closure still references the original `opts` with no `deferralTimeoutMs`, so `maxWaitMs` resolves to `undefined` and the deferral poll runs indefinitely — the same bug this PR is fixing.

To close the gap, consider storing `deferralTimeoutMs` as module-level state (alongside `pendingRestartEmitHooks`) and reading it from that state inside the `setTimeout` callback, rather than from a captured closure, so a coalesced call can override it.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/tools/gateway-tool.ts
Line: 24-25

Comment:
**Duplicate constant risks silent divergence**

`EXPLICIT_RESTART_DEFERRAL_TIMEOUT_MS = 30_000` is independently defined here and again in `src/auto-reply/reply/commands-session.ts`. The two values must stay in sync to provide consistent deferral-timeout protection across explicit restart paths. Exporting a single named constant from `src/infra/restart.ts` and importing it in both call sites would prevent silent drift.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(gateway): bound explicit restart def..." | Re-trigger Greptile

Comment on lines 24 to 25
const log = createSubsystemLogger("gateway-tool");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Duplicate constant risks silent divergence

EXPLICIT_RESTART_DEFERRAL_TIMEOUT_MS = 30_000 is independently defined here and again in src/auto-reply/reply/commands-session.ts. The two values must stay in sync to provide consistent deferral-timeout protection across explicit restart paths. Exporting a single named constant from src/infra/restart.ts and importing it in both call sites would prevent silent drift.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/tools/gateway-tool.ts
Line: 24-25

Comment:
**Duplicate constant risks silent divergence**

`EXPLICIT_RESTART_DEFERRAL_TIMEOUT_MS = 30_000` is independently defined here and again in `src/auto-reply/reply/commands-session.ts`. The two values must stay in sync to provide consistent deferral-timeout protection across explicit restart paths. Exporting a single named constant from `src/infra/restart.ts` and importing it in both call sites would prevent silent drift.

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling gateway Gateway runtime size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant