perf: bound async transcript history reads by steipete · Pull Request #75977 · openclaw/openclaw

steipete · 2026-05-02T06:29:21Z

No description provided.

steipete · 2026-05-02T06:29:29Z

clawsweeper · 2026-05-02T06:30:15Z

🦞🦞
ClawSweeper merged this PR after the passing review.

Source: clawsweeper[bot]
Feedback: structured ClawSweeper verdict: pass (sha=ccfe33658c08c21d2a7014ff1cec9fc70f77e05f)
Merge status: merged by ClawSweeper automerge
Merged at: 2026-05-02T06:36:02Z
Merge commit: 4d9c658f4058

What merged:

The PR bounds async transcript history reads and shares async transcript-index builds across gateway history, embedded/TUI history, restart recovery, fork token checks, and preflight compaction paths.
Reproducibility: not applicable. this is a performance PR rather than a user bug report. The verification pa ... ource review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing.

Fixups included:

No separate fixup commits were needed after automerge opt-in.

The automerge loop is complete.

Automerge progress:

2026-05-02 06:30:14 UTC review queued [`ccfe33658c08`](https://github.com/openclaw/openclaw/commit/ccfe33658c08c21d2a7014ff1cec9fc70f77e05f) (queued)
2026-05-02 06:34:37 UTC review passed [`ccfe33658c08`](https://github.com/openclaw/openclaw/commit/ccfe33658c08c21d2a7014ff1cec9fc70f77e05f) (structured ClawSweeper verdict: pass (sha=ccfe33658c08c21d2a7014ff1cec9fc70f77e...)
2026-05-02 06:36:04 UTC merged [`ccfe33658c08`](https://github.com/openclaw/openclaw/commit/ccfe33658c08c21d2a7014ff1cec9fc70f77e05f) (merged by ClawSweeper automerge)

clawsweeper · 2026-05-02T06:31:42Z

Codex review: passed.

Summary
The PR bounds async transcript history reads and shares async transcript-index builds across gateway history, embedded/TUI history, restart recovery, fork token checks, and preflight compaction paths.

Reproducibility: not applicable. this is a performance PR rather than a user bug report. The verification path is source review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing.

Next step before merge
No repair lane is needed; this automerge-opted PR has no actionable review findings, and exact-head checks can gate merge.

Security
Cleared: The diff changes TypeScript transcript/history logic, tests, docs, and changelog only, with no dependency, workflow, secret, or code-download surface added.

Review details

Best possible solution:

Let the automerge path proceed only after exact-head CI and mergeability finish cleanly, keeping the explicit full-vs-recent transcript read contract and focused tests.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a performance PR rather than a user bug report. The verification path is source review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing.

Is this the best way to solve the issue?

Yes; the patch keeps full transcript scans explicit for surfaces that need them while moving first-page and recovery-style paths to bounded reads, which is the narrow maintainable fix for the observed hot path.

What I checked:

Discussion and automerge context: The issue comments show @clawsweeper automerge, and the bot enabled exact-head automerge review for head ccfe33658c08c21d2a7014ff1cec9fc70f77e05f. (ccfe33658c08)
Async read API now distinguishes bounded and full scans: readSessionMessagesAsync requires an explicit mode; recent mode delegates to the bounded tail reader, while full mode still uses the transcript index. (src/gateway/session-utils.fs.ts:481, ccfe33658c08)
Concurrent index builds are coalesced: The async transcript index now records in-flight builds by file path plus mtimeMs and size, and clears them after the shared promise resolves. (src/gateway/session-transcript-index.fs.ts:245, ccfe33658c08)
Embedded and TUI history reads are bounded: Embedded gateway history and TUI history both read recent transcript windows sized from the requested limit and chat-history byte budget before projection. (src/tui/embedded-backend.ts:199, ccfe33658c08)
Compaction and fork token checks avoid full message materialization: Preflight compaction uses transcript usage/byte snapshots first and falls back to bounded recent message reads; parent fork checks use recent usage plus conservative byte and cached-token estimates. (src/auto-reply/reply/session-fork.runtime.ts:76, ccfe33658c08)
Focused regression coverage exists: The PR adds tests for shared concurrent index builds, recent-mode byte caps, aggregate recent usage, and latest recent usage extraction. (src/gateway/session-utils.fs.test.ts:830, ccfe33658c08)

Likely related people:

steipete: Recent history shows Peter authored the async transcript IO change and several adjacent transcript, memory compaction, and fork-session refactors that this PR builds on. (role: recent maintainer and primary adjacent owner; confidence: high; commits: 6147e1b91d3e, 7dc5b9484f73, 9a814bcec242; files: src/gateway/session-utils.fs.ts, src/gateway/session-transcript-index.fs.ts, src/auto-reply/reply/agent-runner-memory.ts)
vincentkoc: Vincent recently worked on bounding chat-history and session transcript hot paths, which are directly adjacent to this PR's read-window changes. (role: bounded transcript hot-path contributor; confidence: medium; commits: aec83af23d49, ec59af33863a, ecf6cbf75d3d; files: src/gateway/session-utils.fs.ts, src/gateway/server-methods/chat.ts, src/auto-reply/reply/agent-runner-memory.ts)

Remaining risk / open question:

One exact-head check, checks-node-agentic-control-plane-runtime, was still in progress at review time, so merge should remain gated on final required CI status.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 8adbee3a686a.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccfe33658c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-02T06:34:52Z

    if (messages.length === 0) {
      return undefined;


Preserve byte fallback when tail read returns no messages

If the bounded recent read hits a huge trailing JSONL record (or otherwise yields zero parseable messages), this early return undefined drops the already-computed transcriptBytesTokens fallback from readSessionLogSnapshot. In stale sessions without usage metadata, runPreflightCompactionIfNeeded then sees no token estimate and can skip compaction even when the transcript is large, which is a regression from the previous full-read behavior. Return a byte-based estimate here instead of undefined when snapshot.byteSize is available.

Useful? React with 👍 / 👎.

Summary: - The PR bounds async transcript history reads and shares async transcript-index builds across gateway history, embedded/TUI history, restart recovery, fork token checks, and preflight compaction paths. - Reproducibility: not applicable. this is a performance PR rather than a user bug report. The verification pa ... ource review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing. ClawSweeper fixups: - No separate fixup commits were needed after automerge opt-in. Validation: - ClawSweeper review passed for head ccfe336. - Required merge gates passed before the squash merge. Prepared head SHA: ccfe336 Review: openclaw#75977 (comment) Co-authored-by: Peter Steinberger <[email protected]>

Summary: - The PR bounds async transcript history reads and shares async transcript-index builds across gateway history, embedded/TUI history, restart recovery, fork token checks, and preflight compaction paths. - Reproducibility: not applicable. this is a performance PR rather than a user bug report. The verification pa ... ource review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing. ClawSweeper fixups: - No separate fixup commits were needed after automerge opt-in. Validation: - ClawSweeper review passed for head ccfe33658c08c21d2a7014ff1cec9fc70f77e05f. - Required merge gates passed before the squash merge. Prepared head SHA: ccfe33658c08c21d2a7014ff1cec9fc70f77e05f Review: openclaw/openclaw#75977 (comment) Co-authored-by: Peter Steinberger <[email protected]>

Summary: - The PR bounds async transcript history reads and shares async transcript-index builds across gateway history, embedded/TUI history, restart recovery, fork token checks, and preflight compaction paths. - Reproducibility: not applicable. this is a performance PR rather than a user bug report. The verification pa ... ource review plus the added unit coverage for bounded reads, usage snapshots, and concurrent index sharing. ClawSweeper fixups: - No separate fixup commits were needed after automerge opt-in. Validation: - ClawSweeper review passed for head ccfe336. - Required merge gates passed before the squash merge. Prepared head SHA: ccfe336 Review: openclaw#75977 (comment) Co-authored-by: Peter Steinberger <[email protected]>

perf: bound async transcript history reads

ccfe336

openclaw-barnacle Bot added docs Improvements or additions to documentation gateway Gateway runtime agents Agent runtime and tooling size: XL maintainer Maintainer-authored PR labels May 2, 2026

clawsweeper Bot added the clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge label May 2, 2026

chatgpt-codex-connector Bot reviewed May 2, 2026

View reviewed changes

clawsweeper Bot merged commit 4d9c658 into main May 2, 2026
108 of 110 checks passed

clawsweeper Bot deleted the perf/async-transcript-history-cache branch May 2, 2026 06:36

clawsweeper Bot mentioned this pull request May 2, 2026

fix(sessions): fall back to latest reset archive for missing async transcripts #76134

Closed

22 tasks

clawsweeper Bot mentioned this pull request May 10, 2026

[Feature]: logical chat history across rolled sessions #43929

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf: bound async transcript history reads#75977

perf: bound async transcript history reads#75977
clawsweeper[bot] merged 1 commit into
mainfrom
perf/async-transcript-history-cache

steipete commented May 2, 2026

Uh oh!

steipete commented May 2, 2026

Uh oh!

clawsweeper Bot commented May 2, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 2, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

steipete commented May 2, 2026

Uh oh!

steipete commented May 2, 2026

Uh oh!

clawsweeper Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

clawsweeper Bot commented May 2, 2026 •

edited

Loading

clawsweeper Bot commented May 2, 2026 •

edited

Loading