Skip to content

fix(sessions): complete tail-read windows despite short positional reads#108655

Merged
steipete merged 2 commits into
openclaw:mainfrom
Monkey-wusky:fix/session-utils-short-read
Jul 16, 2026
Merged

fix(sessions): complete tail-read windows despite short positional reads#108655
steipete merged 2 commits into
openclaw:mainfrom
Monkey-wusky:fix/session-utils-short-read

Conversation

@Monkey-wusky

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where readRecentSessionMessagesAsync and readLastMessagePreviewFromOpenTranscriptAsync could silently return incomplete tail data when a positional handle.read() returned fewer bytes than requested — a documented POSIX behavior, common on network filesystems (NFS, FUSE) and container bind mounts.

On main, both paths call handle.read(buffer, 0, readLen, readStart) once and use bytesRead as-is. When the read returns short, the remaining buffer bytes are garbage and the caller surfaces truncated messages or null previews without any error.

The same short-read anti-pattern was recently fixed by sunlit-deng in:

Why This Change Was Made

Both tail-read paths now call the existing readFileWindowFully helper instead of a single-shot handle.read(). The helper loops until the requested buffer window fills or the file reaches EOF. This is a mechanical substitution — the readStart, readLen, and all downstream line-splitting and filtering logic are unchanged.

User Impact

Session history reads and last-message previews return complete data on network filesystems. There is no API shape or configuration change, no additional I/O on local filesystems (the loop executes exactly once), and no change to byte/line caps.

Evidence

On current main, a proof script capping each handle.read() at 16 bytes to simulate NFS reproduces the short read:

File: 8465 bytes, reading last 512 bytes from offset 7953

--- Simulated NFS (cap each read at 16 bytes) ---

OLD (single handle.read):  requested=512  got=16
  ❌ Only 16/512 bytes read — remaining 496 bytes = garbage

NEW (readFileWindowFully): requested=512  got=512
  ✅ 32 iterations x 16-byte reads — buffer fully filled

With this change, the same short-read proxy produces the same result as an uncapped local read, validated by two new vitest tests:

 ✓ readRecentSessionMessagesAsync survives 16-byte tail read caps
 ✓ readRecentSessionMessagesAsync honors maxBytes under short reads

The complete owning test file also passes, including the existing title-fields, usage, preview, and oversized-line coverage:

Test Files  3 passed (3)
Tests       12 passed | 186 skipped (198)

@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S labels Jul 16, 2026
@steipete steipete self-assigned this Jul 16, 2026
@clawsweeper

clawsweeper Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 16, 2026, 2:10 AM ET / 06:10 UTC.

Summary
Replaces two single-shot positional transcript reads with readFileWindowFully and adds short-read regression coverage for recent session messages.

PR surface: Source +1, Tests +84. Total +85 across 2 files.

Reproducibility: yes. from source and deterministic short-read injection: current main performs one positional read, while the branch’s mocked production-path test caps reads at 16 bytes and exercises the failure contract. A real NFS or production-entrypoint run was not supplied.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: serialized state: src/gateway/session-utils.fs.test.ts, unknown-data-model-change: src/gateway/session-utils.fs.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Move the short-read tests into a focused companion test file and resolve the session-accessor boundary check.
  • [P1] Add a short-read regression case for readLastMessagePreviewFromOpenTranscriptAsync.
  • [P1] Post redacted terminal output exercising the real recent-message and preview entrypoints before and after the fix.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The terminal output demonstrates the helper against a real file, but not either changed production session entrypoint; add redacted live output for recent-message and preview reads, then update the PR body to trigger another review or ask a maintainer to comment @clawsweeper re-review.

Risk before merge

  • [P1] The current head still has failed checks-fast-max-lines-ratchet and check-session-accessor-boundary checks; only the former can be diagnosed conclusively from the supplied context.
  • [P1] The helper-level terminal output does not prove that either changed production session entrypoint returns the expected complete result after the fix, and the last-message preview path has no dedicated short-read regression test.

Maintainer options:

  1. Decide the mitigation before merge
    Keep the shared-helper substitution, move the short-read coverage into a focused companion test file, cover the preview path, and provide redacted terminal output invoking the real recent-message and preview entrypoints under capped positional reads.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] Contributor follow-up is needed for the failed repository gates and real-behavior proof; the proof requirement cannot be completed safely by an automated repair worker.

Security
Cleared: The diff reuses an existing local filesystem-read helper and adds tests without changing dependencies, secrets, permissions, package resolution, or executable supply-chain surfaces.

Review findings

  • [P2] Move the new coverage out of the ratcheted test file — src/gateway/session-utils.fs.test.ts:2039
Review details

Best possible solution:

Keep the shared-helper substitution, move the short-read coverage into a focused companion test file, cover the preview path, and provide redacted terminal output invoking the real recent-message and preview entrypoints under capped positional reads.

Do we have a high-confidence way to reproduce the issue?

Yes from source and deterministic short-read injection: current main performs one positional read, while the branch’s mocked production-path test caps reads at 16 bytes and exercises the failure contract. A real NFS or production-entrypoint run was not supplied.

Is this the best way to solve the issue?

Yes for the implementation boundary: reusing the existing full-window helper is narrower and more maintainable than duplicating a loop. The branch still needs its tests relocated, preview coverage, and stronger entrypoint-level proof before merge.

Full review comments:

  • [P2] Move the new coverage out of the ratcheted test file — src/gateway/session-utils.fs.test.ts:2039
    This added block pushes src/gateway/session-utils.fs.test.ts past the repository’s max-lines ratchet, and checks-fast-max-lines-ratchet consequently fails on this head. Move these focused short-read cases into a companion test file, or otherwise reduce the owning file below the enforced limit, so the branch can pass the required gate.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 5f2828b50ada.

Label changes

Label changes:

  • add P2: This is a bounded session-history correctness fix for filesystems that legally return short positional reads, with limited but user-visible transcript impact.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.

Label justifications:

  • P2: This is a bounded session-history correctness fix for filesystems that legally return short positional reads, with limited but user-visible transcript impact.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
Evidence reviewed

PR surface:

Source +1, Tests +84. Total +85 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 3 2 +1
Tests 1 84 0 +84
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 87 2 +85

What I checked:

Likely related people:

  • Vincent Koc: Current-main history for the bounded recent transcript reader and related session-history projections points to commit 33e19fb5aeead4842ab5a4afd7ce28012a571bec. (role: introduced current session-history behavior; confidence: high; commits: 33e19fb5aeea; files: src/gateway/session-utils.fs.ts, src/gateway/server-methods/chat.ts, src/gateway/sessions-history-http.ts)
  • sunlit-deng: Authored the merged asynchronous full-window helper change and the neighboring synchronous sessions-tail short-read fix that establish the intended contract. (role: introduced helper and adjacent fixes; confidence: high; commits: 8cc8feee4d35, 55a822dc6877; files: src/infra/file-read.ts, src/agents/bootstrap-files.ts, src/commands/sessions-tail.ts)
  • steipete: Recent gateway session-history refactors and display-projection work touch the same utility and caller surfaces, making this a useful routing candidate for integration review. (role: recent adjacent area contributor; confidence: medium; commits: 079bf25fee2c, f9108120c2a3, bcd61f0a382d; files: src/gateway/session-utils.fs.ts, src/gateway/server-methods/chat.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete force-pushed the fix/session-utils-short-read branch 2 times, most recently from 164fe76 to 32775c1 Compare July 16, 2026 09:43
Monkey-wusky and others added 2 commits July 16, 2026 10:55
Replace single-shot handle.read() calls in two session tail-read
paths with readFileWindowFully so multibyte-range positional reads
do not silently return incomplete tail data on network filesystems.

- readRecentTranscriptTailLinesAsync: tail-window read for recent
  session messages now loops until the requested window fills
- readLastMessagePreviewFromOpenTranscriptAsync: last-message
  preview tail read now loops until the requested window fills

The same readFileWindowFully helper was introduced in openclaw#108253 and
expanded with a sync variant in openclaw#108127 (both by sunlit-deng).
@steipete
steipete force-pushed the fix/session-utils-short-read branch from 32775c1 to 031f724 Compare July 16, 2026 09:57
@steipete

Copy link
Copy Markdown
Contributor

Maintainer pass completed the short-read repair across every sibling path.

Node's file-read contract permits fewer bytes than requested, so each caller must use the returned byte count and continue until its window is filled or EOF is reached: https://nodejs.org/api/fs.html#fsreadfd-buffer-offset-length-position-callback

@steipete
steipete merged commit 8ebb1d2 into openclaw:main Jul 16, 2026
115 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Merged via squash.

@Monkey-wusky
Monkey-wusky deleted the fix/session-utils-short-read branch July 16, 2026 10:57
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 17, 2026
…ads (openclaw#108655)

* fix(sessions): complete tail-read windows despite short positional reads

Replace single-shot handle.read() calls in two session tail-read
paths with readFileWindowFully so multibyte-range positional reads
do not silently return incomplete tail data on network filesystems.

- readRecentTranscriptTailLinesAsync: tail-window read for recent
  session messages now loops until the requested window fills
- readLastMessagePreviewFromOpenTranscriptAsync: last-message
  preview tail read now loops until the requested window fills

The same readFileWindowFully helper was introduced in openclaw#108253 and
expanded with a sync variant in openclaw#108127 (both by sunlit-deng).

* fix(sessions): complete sync title preview reads

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants