Skip to content

fix(transcripts): close stream on parse error in readUtterancesFromDir (fixes #98467)#98584

Closed
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/transcripts-fd-leak-readUtterancesFromDir
Closed

fix(transcripts): close stream on parse error in readUtterancesFromDir (fixes #98467)#98584
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/transcripts-fd-leak-readUtterancesFromDir

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What Problem This Solves

TranscriptsStore.readUtterancesFromDir creates a createReadStream and readline.createInterface inline inside a try block for streaming JSONL parsing. If JSON.parse throws on a malformed line, the catch block re-throws the error but the stream and readline interface are never cleaned up — the underlying file descriptor remains open until garbage collection.

In long-running gateway processes that parse many transcript sessions, leaked file descriptors accumulate and can exhaust the OS fd limit.

Why This Change Was Made

The stream and readline references were created inline inside the try block, making them unreachable from catch or any cleanup path. By extracting them to outer-scope variables and adding a finally block that calls lines.close() and stream.destroy(), the file descriptor is released immediately on all exit paths — normal completion, parse error, or ENOENT.

User Impact

Operators running OpenClaw gateways with active transcript sessions will no longer accumulate leaked file descriptors when malformed JSONL lines are encountered. This prevents potential EMFILE errors in long-running processes.

Evidence

  • Behavior addressed: File descriptor leak in readUtterancesFromDir when JSON.parse throws on malformed JSONL
  • Environment tested: macOS, Node.js v22.22.3, OpenClaw 2026.6.11 (5d52b5b)
  • Steps run after the patch: Created a test JSONL file with valid lines + one malformed line, exercised the streaming parse path with try/finally cleanup
  • Evidence after fix: Stream is properly destroyed after parse error:
Parse error caught: Unexpected token 'N', "NOT_VALID_JSON{{{" is not valid JSON
finally: lines.close() + stream.destroy() called
Stream destroyed: true
Stream readable: false
File unlinked successfully (FD released)
Test PASSED: stream cleanup on parse error works correctly
  • Observed result after fix: stream.destroyed === true, stream.readable === false, file descriptor released immediately
  • What was not tested: Production gateway with real transcript accumulation over time; only single-file parse-error path verified

Real behavior proof

  • Behavior addressed: File descriptor leak in readUtterancesFromDir when JSON.parse throws on malformed JSONL
  • Environment tested: macOS, Node.js v22.22.3, OpenClaw 2026.6.11 (5d52b5b)
  • Steps run after the patch: Created a test JSONL file with valid lines + one malformed line, exercised the streaming parse path with try/finally cleanup
  • Evidence after fix: Stream is properly destroyed after parse error:
Parse error caught: Unexpected token 'N', "NOT_VALID_JSON{{{" is not valid JSON
finally: lines.close() + stream.destroy() called
Stream destroyed: true
Stream readable: false
File unlinked successfully (FD released)
Test PASSED: stream cleanup on parse error works correctly
  • Observed result after fix: stream.destroyed === true, stream.readable === false, file descriptor released immediately
  • What was not tested: Production gateway with real transcript accumulation over time; only single-file parse-error path verified

Changes Made

  • src/transcripts/store.ts: Extract createReadStream and createInterface to outer scope variables; add finally block with lines.close() and stream.destroy() to ensure cleanup on all exit paths

  • AI-assisted (Hermes Agent)

Wrap the streaming JSONL loop in try/finally to call lines.close() and
stream.destroy() on all exit paths. Previously, if JSON.parse threw
inside the for-await loop, the underlying file descriptor remained open
because the createReadStream and createInterface references were scoped
inside the try block and unreachable from the catch.

Fixes openclaw#98467
@vincentkoc

Copy link
Copy Markdown
Member

Thanks for the fix here. The transcript parse-error stream cleanup is now covered by the stronger canonical fix in #98493, which landed as 94cb14b. Closing this PR to keep the queue clean because #98493 includes focused regression coverage and explicit stream/readline cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants