fix: page sessions_history beyond truncated tails by galiniliev · Pull Request #97101 · openclaw/openclaw

galiniliev · 2026-06-26T20:11:37Z

Related: #90981

What Problem This Solves

Fixes an issue where parent agents using sessions_history could only inspect the newest bounded tail of long child sessions. When that tail was truncated or started inside late-session context, there was no supported way to page backward through older transcript windows without reading raw JSONL files.

Why This Change Was Made

This adds explicit offset pagination to sessions_history and Gateway chat.history, returning offset, nextOffset, hasMore, and totalMessages metadata for explicit offset requests. The implementation keeps the existing no-offset newest-tail behavior, keeps display sanitization and byte caps, computes continuation from the final returned page boundary, overreads one context row for stale announce-pair filtering, and keeps offset pages scoped to OpenClaw transcript windows instead of merging whole external CLI fallback imports into every page.

This does not add a readable full-history export surface; #90981 should remain open for that broader product/API decision if maintainers still want it.

User Impact

Agents and operators can now call sessions_history with offset: 0, then follow nextOffset to page backward through long child-session transcripts. This makes recovery and audit workflows deterministic while preserving the bounded, redacted history view.

Evidence

Before evidence:

2026-06-26T02:36:12.812Z CALL sessions_history limit=3 for multiple child sessions
2026-06-26T02:36:17.234Z CALL sessions_history limit=2 for the affected child session
2026-06-26T02:36:19.990Z CALL sessions_history limit=1 for the affected child session
2026-06-26T02:36:23.101Z CALL sessions_history includeTools=false limit=5 for the affected child session

Observed result: calls stayed anchored to newest tail slices and exposed no cursor/offset to retrieve older omitted windows.

Behavior addressed:

sessions_history and Gateway chat.history now support explicit offset pagination for bounded, sanitized OpenClaw transcript windows.

Real environment tested:

Local OpenClaw source checkout in a Codex worktree on Linux/WSL2, Node/pnpm dependencies already installed. Real live recovery logs from June 26, 2026 were used as before-fix evidence; no live provider replay was run after the patch because the bug is in local transcript pagination/schema handling before model generation.

Exact steps or command run after this patch:

node scripts/run-vitest.mjs src/agents/tools/sessions-history-tool.test.ts src/agents/tools/embedded-gateway-stub.test.ts packages/gateway-protocol/src/index.test.ts src/gateway/server.chat.gateway-server-chat-b.test.ts
node scripts/format-docs.mjs --check docs/concepts/session-tool.md docs/concepts/delegate-architecture.md docs/tools/subagents.md
node scripts/run-tsgo.mjs -p tsconfig.core.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core.tsbuildinfo
node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.src.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/test-src.tsbuildinfo
node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.packages.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/test-packages.tsbuildinfo
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode local

Evidence after fix:

[test] passed 3 Vitest shards in 51.59s
Docs formatting clean (669 files).
tsgo core: passed
tsgo test src: passed
tsgo test packages: passed
git diff --check: passed
autoreview clean: no accepted/actionable findings reported
overall: patch is correct (0.78)

Observed result after fix:

Focused tests cover offset validation and forwarding, first-page nextOffset, nonzero page reads, stale announce-pair boundary overread, final byte-budget cursor calculation, embedded gateway parity, protocol schema acceptance, and docs/tool metadata updates. The final autoreview found no accepted/actionable findings.

Follow-up evidence for the P1 review finding:

commit 9800182da19f52c118671608f84f0ba19546b34b recomputes sessions_history pagination from the final tool-visible messages after includeTools filtering, sanitization, and the 80 KiB tool cap. It also preserves __openclaw.seq on the oversized-message placeholder so the cursor still advances accurately when the last surviving row is replaced.

node scripts/run-vitest.mjs src/agents/tools/sessions-history-tool.test.ts src/agents/tools/embedded-gateway-stub.test.ts packages/gateway-protocol/src/index.test.ts src/gateway/server.chat.gateway-server-chat-b.test.ts
[test] passed 3 Vitest shards in 65.91s

.agents/skills/autoreview/scripts/autoreview --mode local
autoreview clean: no accepted/actionable findings reported
overall: patch is correct (0.87)

Azure Crabbox real-history proof for the P1 review finding:

provider=azure lease=cbx_ecf9405b4cb2 slug=amber-hermit
remote=/work/crabbox/cbx_ecf9405b4cb2/bug-046-sessions-history-pagination
sha=c25348abfed6ecb1a4d0206936f1ea36ad502d01
node=v22.23.1 pnpm=11.2.2

[test] passed 3 Vitest shards in 36.60s
unit packages/gateway-protocol/src/index.test.ts: 45 passed
gateway src/gateway/server.chat.gateway-server-chat-b.test.ts: 88 passed
agents src/agents/tools/sessions-history-tool.test.ts + embedded-gateway-stub.test.ts: 25 passed

AFTER_FIX_SEEDED_TRANSCRIPT totalRows=64 sessionKey=main stateDir=<redacted> transcript=<redacted>
AFTER_FIX_PAGE_1 offset=0 count=8 seqs=57,58,59,60,61,62,63,64 labels=proof-turn-57,proof-turn-58,proof-turn-59,proof-turn-60,proof-turn-61,proof-turn-62,proof-turn-63,proof-turn-64 nextOffset=8 hasMore=true totalMessages=64
AFTER_FIX_PAGE_2 offset=8 count=8 seqs=49,50,51,52,53,54,55,56 labels=proof-turn-49,proof-turn-50,proof-turn-51,proof-turn-52,proof-turn-53,proof-turn-54,proof-turn-55,proof-turn-56 nextOffset=16 hasMore=true totalMessages=64
AFTER_FIX_PAGE_3 offset=16 count=8 seqs=41,42,43,44,45,46,47,48 labels=proof-turn-41,proof-turn-42,proof-turn-43,proof-turn-44,proof-turn-45,proof-turn-46,proof-turn-47,proof-turn-48 nextOffset=24 hasMore=true totalMessages=64
AFTER_FIX_PAGE_4 offset=24 count=8 seqs=33,34,35,36,37,38,39,40 labels=proof-turn-33,proof-turn-34,proof-turn-35,proof-turn-36,proof-turn-37,proof-turn-38,proof-turn-39,proof-turn-40 nextOffset=32 hasMore=true totalMessages=64
AFTER_FIX_PAGE_5 offset=32 count=8 seqs=25,26,27,28,29,30,31,32 labels=proof-turn-25,proof-turn-26,proof-turn-27,proof-turn-28,proof-turn-29,proof-turn-30,proof-turn-31,proof-turn-32 nextOffset=40 hasMore=true totalMessages=64
AFTER_FIX_PROOF followedNextOffsetPages=5 pageSeqsMatch=true offsetsMatch=true nextOffsetsMatch=true eachPageAscending=true

The behavior harness seeded an isolated OpenClaw state directory with a 64-row transcript, invoked the real sessions_history tool through embedded Gateway chat.history, followed returned nextOffset values for five pages, and verified each page covered the expected older transcript window without skipping rows.

What was not tested:

No live provider replay was run after the patch because this fix is local transcript pagination behavior before model generation. The requested real transcript pagination behavior was verified on Azure Crabbox with an isolated seeded OpenClaw state directory and the real sessions_history -> embedded Gateway chat.history path.

clawsweeper · 2026-06-26T20:12:42Z

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 6:17 PM ET / 22:17 UTC.

Summary
The PR adds explicit offset pagination to sessions_history and Gateway chat.history, with pagination metadata, protocol/client updates, docs, tests, and prompt snapshot updates.

Reproducibility: yes. at source level: current main exposes no sessions_history or Gateway chat.history offset parameter while bounded output can truncate older transcript rows. I did not run tests because this is a read-only review.

Review metrics: 1 noteworthy metric.

Pagination API surface: 1 request parameter added, 4 response metadata fields surfaced. This changes model-facing and Gateway history contracts, so maintainers should notice the API shape before merge.

Stored data model
Persistent data-model change detected: persistent cache schema: ui/src/i18n/.i18n/th.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/tr.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/uk.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/vi.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/zh-CN.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/zh-TW.tm.jsonl, and 2 more. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Get maintainer sign-off on the bounded offset-pagination contract.
[P2] Let required checks finish on the current head before merge.

Risk before merge

[P1] The PR adds new model-facing and Gateway request/response contract surface that callers may rely on after release, so maintainer API acceptance is still required.
[P1] The offset cursor semantics page persisted transcript windows; the tests and proof are strong, but this remains session-state-sensitive behavior where skipped or repeated rows would be user-visible.
[P1] Live PR metadata reports mergeStateStatus: UNSTABLE; the exact head should finish required checks before any merge decision.

Maintainer options:

Accept Bounded Offset Pagination (recommended)
Maintainers can accept offset, nextOffset, hasMore, and totalMessages as the bounded tool/Gateway contract after exact-head checks are green.
Revise To Cursor-Language API
If maintainers prefer the existing REST history cursor vocabulary, ask for a pre-merge schema, docs, tests, generated-client, and prompt snapshot revision.
Pause For Export Contract
If pagination and readable export must be designed together, pause this PR and settle the broader product/security decision on [Feature] sessions_history: add pagination/offset and export support #90981 first.

Next step before merge

[P2] The remaining action is maintainer review of a protected public history API/session-state contract plus normal CI completion, not an autonomous repair lane.

Security
Cleared: No concrete security regression was found; the patch keeps visibility checks, redaction, bounded output, and does not add dependencies, workflow permissions, secrets handling, or new code execution paths.

Review details

Best possible solution:

Land the bounded offset-pagination subset after maintainer acceptance of the public API shape, while leaving readable full-history export tracked on #90981.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main exposes no sessions_history or Gateway chat.history offset parameter while bounded output can truncate older transcript rows. I did not run tests because this is a read-only review.

Is this the best way to solve the issue?

Mostly yes for the bounded pagination subset: adding pagination at the tool/Gateway layer preserves the sanitized history surface and avoids raw transcript export. The exact public API shape still needs maintainer acceptance, and readable full-history export remains separate.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 36722014efa3.

Label changes

Label justifications:

P2: The PR improves bounded session-history retrieval with limited blast radius rather than addressing an urgent runtime outage.
merge-risk: 🚨 compatibility: The diff adds new tool and Gateway request/response fields that clients and model-facing callers may rely on after release.
merge-risk: 🚨 session-state: The new offset semantics page persisted transcript windows and could skip or repeat session history if the cursor boundary is wrong.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes redacted Azure Crabbox terminal output showing after-fix nextOffset pagination through a seeded transcript via the real sessions_history to embedded Gateway chat.history path.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted Azure Crabbox terminal output showing after-fix nextOffset pagination through a seeded transcript via the real sessions_history to embedded Gateway chat.history path.

Evidence reviewed

What I checked:

Repository policy applied: Root and scoped AGENTS.md guidance was read; session-state, Gateway protocol, docs, scripts, and generated client changes are compatibility-sensitive review surfaces. (AGENTS.md:20, 36722014efa3)
Current main lacks tool offset support: Current main's sessions_history schema accepts sessionKey, limit, and includeTools, then forwards only sessionKey and limit to chat.history. (src/agents/tools/sessions-history-tool.ts:32, 36722014efa3)
Current main lacks Gateway offset support: Current main's Gateway ChatHistoryParamsSchema accepts sessionKey, optional agentId, limit, and maxChars, with no offset parameter. (packages/gateway-protocol/src/schema/logs-chat.ts:30, 36722014efa3)
PR adds bounded tool pagination: The PR head adds offset validation, forwards it to chat.history, and recomputes continuation metadata after filtering, sanitization, and byte caps. (src/agents/tools/sessions-history-tool.ts:38, 0cab379382e8)
PR adds Gateway page reads: The PR head routes explicit offset requests through bounded transcript page reads, projection, byte caps, and nextOffset derivation from the final bounded page. (src/gateway/server-methods/chat.ts:2710, 0cab379382e8)
Transcript index supports seq-based paging: The transcript index numbers visible records and the new page reader slices by offset from newest-visible entries, matching the cursor math used by the PR. (src/gateway/session-utils.fs.ts:761, 0cab379382e8)

Likely related people:

pick-cat: Current-main blame on sessions-history-tool.ts, session-utils.fs.ts, and Gateway protocol schema points to recent session/tool work in commit 4985671. (role: recent area contributor; confidence: medium; commits: 498567190d29; files: src/agents/tools/sessions-history-tool.ts, src/gateway/session-utils.fs.ts, packages/gateway-protocol/src/schema/logs-chat.ts)
Peter Steinberger: Local history shows repeated work on session-history sanitization, Gateway handler structure, CLI history backfill, and agent tool descriptions around the affected history surface. (role: adjacent session and Gateway contributor; confidence: medium; commits: b04dd6d05c3d, 98ea8e244f97, 31f5463a1c5a; files: src/gateway/server-methods/chat.ts, src/gateway/session-utils.fs.ts, src/agents/tools/sessions-history-tool.ts)
Eva: History includes SSE history sanitization and cursor-paging work, which is the closest existing precedent for transcript window pagination. (role: adjacent cursor/history contributor; confidence: medium; commits: dea515e833f2, b0994275703e; files: src/gateway/sessions-history-http.ts, src/gateway/session-history-state.ts)
gut-puncture: Prior ClawSweeper evidence for the canonical issue ties the bounded sessions_history truncation behavior to commit bccdc95. (role: introduced cap behavior; confidence: medium; commits: bccdc95a9b24; files: src/agents/tools/sessions-history-tool.ts, src/agents/openclaw-tools.sessions.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

galiniliev

Manual review confirms the existing pagination blocker.

Finding:

[P1] sessions_history forwards Gateway nextOffset / hasMore after applying its own includeTools filtering, sanitization, and 80 KiB cap (src/agents/tools/sessions-history-tool.ts:296, src/agents/tools/sessions-history-tool.ts:319). The PR correctly moves Gateway continuation metadata after Gateway byte budgeting, but the model-facing tool then returns hardened.items; if that tool cap drops older rows from the page, callers advance past rows they never received. Please recompute continuation from the final tool-visible messages or keep fetching until the tool can return a bounded page with accurate continuation metadata, and add a regression for a capped sessions_history offset page.

Best-fix verdict: close, but not merge-ready. The Gateway-level pagination shape is plausible and well covered; the model-facing tool still needs to make its own pagination metadata match the exact payload it returns.

galiniliev

Manual follow-up review after 9800182da1:

Findings:

[P1] Regenerate protocol clients after adding chat.history.offset — packages/gateway-protocol/src/schema/logs-chat.ts:35
The PR adds offset to ChatHistoryParamsSchema, but the generated protocol artifacts were not committed. CI's checks-fast-bundled-protocol runs pnpm protocol:gen && pnpm protocol:gen:swift && git diff --exit-code ... and now fails because apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift still lacks ChatHistoryParams.offset and its coding key. This leaves native/shared protocol clients out of sync with the Gateway schema. Regenerate and commit the protocol output.
[P1] Fix lint errors in the new gateway pagination tests — src/gateway/server.chat.gateway-server-chat-b.test.ts:3212
CI check-lint fails on the new direct message.__openclaw?.seq assertions at lines 3212, 3230, and 3282 with eslint(no-underscore-dangle). These test assertions need to use a small helper or bracket access so the required lint lane passes.

The earlier pagination-metadata finding is addressed by 9800182da1: sessions_history now recomputes continuation from final tool-visible messages after filtering/capping and preserves seq metadata on the oversized placeholder path. I did not find another runtime correctness blocker in the pagination implementation after reading the Gateway reader/projection path, embedded stub parity path, tool wrapper, docs, and focused tests.

Proof checked: live CI logs for checks-fast-bundled-protocol and check-lint, PR diff against origin/main, scoped AGENTS docs, src/gateway/server-methods/chat.ts, src/gateway/session-utils.fs.ts, src/gateway/session-transcript-readers.ts, src/agents/tools/sessions-history-tool.ts, src/agents/tools/embedded-gateway-stub.ts, and adjacent tests/docs.

galiniliev · 2026-06-27T15:14:34Z

@clawsweeper re-review

clawsweeper · 2026-06-27T15:14:37Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

galiniliev · 2026-06-27T15:40:48Z

@clawsweeper re-review

galiniliev · 2026-06-27T22:30:53Z

Merged via squash.

Prepared head SHA: b4d4c263c6753bbac4f6d6cca6c939267dc5992a
Landed commit: dc575d148a7cb69d0650d271943279a4cd60a7de

openclaw-barnacle Bot added docs Improvements or additions to documentation app: web-ui App: web-ui gateway Gateway runtime agents Agent runtime and tooling size: XL maintainer Maintainer-authored PR labels Jun 26, 2026

galiniliev mentioned this pull request Jun 26, 2026

[Feature] sessions_history: add pagination/offset and export support #90981

Open

galiniliev assigned steipete Jun 26, 2026

galiniliev commented Jun 26, 2026

View reviewed changes

galiniliev assigned galiniliev and unassigned steipete Jun 26, 2026

galiniliev commented Jun 26, 2026

View reviewed changes

clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label Jun 26, 2026

openclaw-barnacle Bot added the scripts Repository scripts label Jun 26, 2026

openclaw-barnacle Bot added channel: qqbot extensions: qa-lab extensions: codex channel: synology-chat extensions: google extensions: ollama extensions: opencode-go extensions: xai labels Jun 27, 2026

galiniliev force-pushed the bug-046-sessions-history-pagination branch from 0cab379 to b4d4c26 Compare June 27, 2026 22:13

github-actions Bot mentioned this pull request Jun 27, 2026

📡 Upstream Digest — 2026-06-27 22:56 UTC curtismercier/openclaw-mods#1162

Open

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(gateway): tools.effective fails for global sessions on non-default agents #97265

Merged

Haderach-Ram mentioned this pull request Jun 28, 2026

Ecosystem Digest — 2026-06-28 Haderach-Ram/openclaw-radar#64

Open

clawsweeper Bot mentioned this pull request Jul 10, 2026

fix(sessions): reject malformed history offsets #103594

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: page sessions_history beyond truncated tails#97101

fix: page sessions_history beyond truncated tails#97101
galiniliev merged 5 commits into
mainfrom
bug-046-sessions-history-pagination

galiniliev commented Jun 26, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

galiniliev left a comment

Uh oh!

galiniliev left a comment

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

galiniliev commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

galiniliev left a comment

Choose a reason for hiding this comment

Uh oh!

galiniliev left a comment

Choose a reason for hiding this comment

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

galiniliev commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

galiniliev commented Jun 26, 2026 •

edited

Loading

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading