Skip to content

fix(gateway): log chat-abort terminal persistence failures#98104

Closed
ly-wang19 wants to merge 3 commits into
openclaw:mainfrom
ly-wang19:fix/chat-abort-persistence-error-log-97795
Closed

fix(gateway): log chat-abort terminal persistence failures#98104
ly-wang19 wants to merge 3 commits into
openclaw:mainfrom
ly-wang19:fix/chat-abort-persistence-error-log-97795

Conversation

@ly-wang19

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes #97795.

When the project session terminal persistence promise inside registerChatAbortController rejected, the .catch() block removed the abort controller without ever recording why. In production this silently masks DB or filesystem health failures during run cleanup — operators get no trace in telemetry or logs that a terminal state failed to persist.

Why This Change Was Made

The bug is a missing telemetry signal, not a logic flaw. The existing behavior (remove the controller either way so memory does not leak) is correct; the only gap is observability. The fix adds a gateway/chat-abort subsystem error log on rejection carrying runId and the underlying error message, then keeps the existing cleanup behavior unchanged.

A subsystem logger is used (matching src/gateway/boot.ts and src/gateway/channel-health-monitor.ts) rather than the bare logError helper so the failure is attributable to this code path in structured logs.

User Impact

  • Operators can now observe chat-abort terminal persistence failures through the standard subsystem logger and correlate them with DB/filesystem health incidents.
  • No behavioral change to run cleanup; abort controllers are still removed on persistence failure so memory does not leak.
  • No gateway runtime or API contract change.

Evidence

  • New test in src/gateway/chat-abort.test.ts: "logs and still removes the controller when terminal persistence rejects". It registers a controller, attaches a rejecting projectSessionTerminalPersistence, calls cleanup(), and asserts both that the entry is removed and that the subsystem logger received error with the expected message and { runId, error } metadata.
  • Existing test "retains completed registrations until terminal persistence succeeds" still covers the success path.
  • Local tests: node scripts/run-vitest.mjs src/gateway/chat-abort.test.ts exits 0.
  • Local typecheck: node scripts/run-tsgo.mjs -p tsconfig.core.json exits 0.

The catch on the terminal persistence promise silently removed the
abort controller without recording why, masking DB/filesystem failures
during run cleanup. Log the rejection via the gateway/chat-abort
subsystem logger before removing the controller so operators can see
the underlying health failure. The controller entry is still removed
either way so memory does not leak.

Closes openclaw#97795.
@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S labels Jun 30, 2026
@clawsweeper

clawsweeper Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close as superseded: the same linked gateway persistence-logging bug is already owned by an open, mergeable, proof-positive canonical PR that logs at the persistence owner boundary with formatter-backed redaction coverage.

Root-cause cluster
Relationship: superseded
Canonical: #97839
Summary: This PR repeats a downstream cleanup-consumer fix for the same linked gateway persistence logging bug, while the canonical open PR is the safer proof-positive owner-boundary landing path.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Canonical path: Close this PR and continue review on #97839 as the canonical owner-boundary fix.

So I’m closing this here and keeping the remaining discussion on #97839.

Review details

Best possible solution:

Close this PR and continue review on #97839 as the canonical owner-boundary fix.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main catches rejected terminal lifecycle persistence in server-chat and downstream cleanup paths without logging; I did not execute tests in this read-only review.

Is this the best way to solve the issue?

No. This branch is a partial downstream mitigation; the better solution is the open canonical owner-boundary PR with formatForLog(err), redaction coverage, and real behavior proof.

Security review:

Security review needs attention: The patch adds an operator-visible arbitrary-error log path without using the canonical gateway formatter and redaction regression proof.

  • [medium] Unformatted persistence error text reaches the new log path — src/gateway/chat-abort.ts:185
    The added log records err.message or String(err) directly in metadata; subsystem sinks redact common secrets, but this bypasses formatForLog(err)'s Error/cause formatting and length-capping path and has no redaction assertion for the new arbitrary persistence-error log.
    Confidence: 0.78

AGENTS.md: found and applied where relevant.

What I checked:

  • Repository policy read: Root AGENTS.md and scoped src/gateway/AGENTS.md were read fully; the owner-boundary, gateway hot-path, and security/logging guidance apply to this review. (AGENTS.md:1, 738b2be4b49b)
  • Current main owner catch is silent: Current main creates the terminal lifecycle persistence promise in server-chat and catches rejection only to broadcast fallback state, which is the owner boundary left unpatched by this PR. (src/gateway/server-chat.ts:714, 738b2be4b49b)
  • Current main downstream cleanup catch is silent: Current main's chat-abort cleanup catch deletes the matching controller after persistence rejection without logging, matching the symptom but not proving the downstream cleanup catch is the best fix point. (src/gateway/chat-abort.ts:176, 738b2be4b49b)
  • Sibling cleanup timing remains outside this PR: When cleanup was requested before terminal persistence is attached, server-runtime-subscriptions consumes the same persistence rejection and deletes or records restart recovery without this PR's new log site running. (src/gateway/server-runtime-subscriptions.ts:101, 738b2be4b49b)
  • Current PR logs downstream raw error metadata: The PR adds logging only inside registerChatAbortController's downstream cleanup catch and records err.message/String(err) as metadata rather than using the gateway formatter path. (src/gateway/chat-abort.ts:183, b651cbff6177)
  • Current PR proof is unit-only: The new test drives the downstream cleanup catch with a mocked subsystem logger and synthetic rejection, but it does not show a real gateway persistence failure path. (src/gateway/chat-abort.test.ts:283, b651cbff6177)

Likely related people:

  • ooiuuii: Authored the merged gateway restart/terminal ownership work that changed server-chat, chat-abort, and server-runtime-subscriptions around the affected persistence path. (role: feature-history contributor; confidence: high; commits: d20fdf3b3842; files: src/gateway/server-chat.ts, src/gateway/chat-abort.ts, src/gateway/server-runtime-subscriptions.ts)
  • steipete: Merged the restart shutdown session marker PR and authored multiple adjacent gateway lifecycle/logging/history commits in the touched area. (role: merger and adjacent lifecycle contributor; confidence: high; commits: d20fdf3b3842, 2b61d38a455c, 4d8502804d70; files: src/gateway/server-chat.ts, src/gateway/chat-abort.ts, src/gateway/ws-log.ts)
  • hexsprite: Authored the gateway ws-log formatter/redaction change that the canonical PR relies on for safer operator-visible error text. (role: adjacent formatter contributor; confidence: medium; commits: 86c1622a3a3c; files: src/gateway/ws-log.ts, src/gateway/ws-log.test.ts)
  • vincentkoc: Recent GitHub path history shows gateway refactors and chat-abort/server-chat touches near the affected lifecycle and abort surfaces. (role: recent adjacent gateway contributor; confidence: medium; commits: 2b75806197ab, ada70ece6fbb, 45e0545e823b; files: src/gateway/server-chat.ts, src/gateway/chat-abort.ts, src/gateway/server-runtime-subscriptions.ts)

Codex review notes: model internal, reasoning high; reviewed against 738b2be4b49b.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. labels Jun 30, 2026
Satisfies oxlint use-unknown-in-catch-callback-variable after the
prior commit introduced the typed catch parameter.
Scan changed paths failed to fetch opengrep from GitHub (infra).
checks-node-compact-large-1 had disposeSession timing flake.
Both unrelated to chat-abort change.
@ly-wang19

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #97839 — that PR fixes the same issue (#97795) at the gateway terminal-persistence owner boundary, which is the cleaner landing path than my chat-abort.ts downstream consumer fix. Thanks for the sweep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Unhandled persistence failures in chat-abort.ts silently swallow errors and risk memory leaks

1 participant