fix(gateway): expose idempotencyKey in chat history metadata by wyf027 · Pull Request #96273 · openclaw/openclaw

wyf027 · 2026-06-24T03:42:50Z

Restored replacement for #94438 after the fork/head repository was recreated.

Summary

carry persisted user-message idempotencyKey into __openclaw transcript metadata during history hydration
preserve the key for live history appends, session.message broadcasts, and oversized history placeholders
fix oversized transcript extraction for the production field order where large content can appear before idempotencyKey
add focused regression coverage for transcript reads, inline history updates, live session message events, and oversized production-order transcript lines

Verification

node_modules/.bin/vitest run --config test/vitest/vitest.gateway.config.ts src/gateway/session-utils.fs.test.ts --reporter=dot
- Test Files 3 passed (3)
- Tests 279 passed (279)
node_modules/.bin/oxfmt --check --threads=1 src/gateway/session-utils.fs.ts src/gateway/session-utils.fs.test.ts
- All matched files use the correct format.
git diff --check
- no output

Real behavior proof

Behavior addressed: Clients can correlate optimistic user bubbles with hydrated or live Gateway messages via __openclaw.idempotencyKey instead of text/FIFO heuristics.

Real environment tested: Local OpenClaw checkout on this PR branch (codex/chat-history-idempotency-key-79844-reopen-20260618, head 14643e647e) running a real loopback Gateway process with a temporary home/state directory. Channels were skipped for isolation, and the Gateway used auth none on 127.0.0.1 only. The placeholder CLI token below only satisfies the CLI URL-override credential guard; it is not a real secret.

Exact steps or command run after this patch:

OPENCLAW_HOME=/tmp/openclaw-pr-96273-proof/home-none \
OPENCLAW_STATE_DIR=/tmp/openclaw-pr-96273-proof/state-none \
OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr-96273-proof/config-none.json \
OPENCLAW_TEST_MINIMAL_GATEWAY=1 OPENCLAW_SKIP_CHANNELS=1 \
node openclaw.mjs gateway run --port 29374 --bind loopback --allow-unconfigured --auth none --force --ws-log compact

OPENCLAW_HOME=/tmp/openclaw-pr-96273-proof/home-none-call \
OPENCLAW_STATE_DIR=/tmp/openclaw-pr-96273-proof/state-none-call \
node openclaw.mjs gateway call chat.send \
  --url ws://127.0.0.1:29374 \
  --token placeholder \
  --timeout 30000 \
  --json \
  --params '{"sessionKey":"main","message":"/context list","idempotencyKey":"proof-96273-idem"}'

OPENCLAW_HOME=/tmp/openclaw-pr-96273-proof/home-none-call \
OPENCLAW_STATE_DIR=/tmp/openclaw-pr-96273-proof/state-none-call \
node openclaw.mjs gateway call agent.wait \
  --url ws://127.0.0.1:29374 \
  --token placeholder \
  --timeout 30000 \
  --json \
  --params '{"runId":"proof-96273-idem","timeoutMs":15000}'

OPENCLAW_HOME=/tmp/openclaw-pr-96273-proof/home-none-call2 \
OPENCLAW_STATE_DIR=/tmp/openclaw-pr-96273-proof/state-none-call2 \
node openclaw.mjs gateway call chat.history \
  --url ws://127.0.0.1:29374 \
  --token placeholder \
  --timeout 30000 \
  --json \
  --params '{"sessionKey":"main"}'

Evidence after fix: Terminal output from the real Gateway RPC flow showed chat.send start, agent.wait complete, and chat.history returning __openclaw.idempotencyKey metadata. The chat.history payload below is trimmed to the relevant fields only.

{
  "runId": "proof-96273-idem",
  "status": "started"
}

{
  "runId": "proof-96273-idem",
  "status": "ok",
  "endedAt": 1782646473456
}

{
  "sessionKey": "main",
  "sessionId": "9f903a13-93c6-4e2f-9de7-60f540ab4862",
  "messages": [
    {
      "role": "user",
      "content": "/context list",
      "idempotencyKey": "proof-96273-idem:user",
      "__openclaw": {
        "id": "5ddebe51-952a-43b1-8a73-98bd2e8d3ffd",
        "idempotencyKey": "proof-96273-idem:user",
        "seq": 1
      }
    },
    {
      "role": "assistant",
      "idempotencyKey": "proof-96273-idem",
      "model": "gateway-injected",
      "__openclaw": {
        "id": "9e1d8024-44a5-41fe-9a07-22f924ccdbeb",
        "idempotencyKey": "proof-96273-idem",
        "seq": 2
      }
    }
  ],
  "sessionInfo": {
    "hasActiveRun": false,
    "lastChannel": "webchat"
  }
}

Observed result after fix: A real local Gateway accepted chat.send, completed the slash-command run, and returned hydrated chat.history messages with the persisted idempotency keys in both the direct message field and __openclaw.idempotencyKey. The oversized production-order case is covered by the updated session-utils.fs.test.ts regression, which now writes content before idempotencyKey and passed in the full file run above.

What was not tested: Native/mobile/web UI rendering was not exercised, and no external model call was made; /context list completed through Gateway's injected command path without provider credentials. The live proof covers the real Gateway RPC/history surface, while the oversized large-content edge case is covered by focused Gateway regression tests.

Reopened after active-PR limit cleanup. Supersedes closed PR #94398.

What Problem This Solves

Clients that optimistically render user messages need the persisted user-message idempotency key to appear in hydrated chat.history responses and live Gateway session.message metadata. Without that key in __openclaw, clients have to fall back to text ordering or FIFO heuristics to correlate optimistic bubbles with server messages.

Evidence

After this patch, a real local Gateway chat.send -> agent.wait -> chat.history flow shows the persisted user idempotencyKey being surfaced in hydrated __openclaw metadata. Focused Gateway tests also cover inline SSE history state, session.message metadata, and oversized production-order transcript lines. Local validation ran formatting and git diff --check successfully.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06166c700d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

wyf027 · 2026-06-24T03:56:03Z

Fixed by preserving idempotencyKey on oversized transcript placeholders and added coverage. Targeted gateway test passed.

clawsweeper · 2026-06-24T05:08:54Z

Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 8:05 AM ET / 12:05 UTC.

Summary
The branch carries persisted Gateway chat idempotency keys into __openclaw.idempotencyKey for hydrated history, live history appends, session.message broadcasts, and oversized placeholders, with regression tests.

PR surface: Source +87, Tests +80. Total +167 across 7 files.

Reproducibility: yes. From source, current main persists the user-turn idempotencyKey but omits it from __openclaw in hydrated history, inline history appends, and session.message metadata; the PR body also supplies after-fix loopback Gateway terminal proof.

Review metrics: 1 noteworthy metric.

Gateway metadata surface: 1 optional __openclaw field added. The PR adds a client-visible chat history and session-message metadata field, which maintainers should notice even though the change is additive.

Stored data model
Persistent data-model change detected: serialized state: src/gateway/session-history-state.test.ts, serialized state: src/gateway/session-history-state.ts, serialized state: src/gateway/session-utils.fs.test.ts, serialized state: src/gateway/session-utils.fs.ts, unknown-data-model-change: src/gateway/session-utils.fs.test.ts, unknown-data-model-change: src/gateway/session-utils.fs.ts, and 4 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #79844
Summary: This PR is the active candidate fix for the canonical Gateway chat.history idempotency metadata issue; earlier implementation attempts are closed unmerged.

Members:

canonical: chat.history: surface idempotencyKey on hydrated user messages for client-side correlation #79844 - The issue describes the same missing __openclaw.idempotencyKey metadata and this PR uses closing syntax for it.
superseded: fix(gateway): expose idempotencyKey in chat history metadata #94438 - This closed unmerged earlier same-author branch had the same intended Gateway metadata fix and the current PR says it replaces it after the fork/head repository was recreated.
superseded: fix(gateway): expose idempotencyKey in chat history metadata #94398 - This closed unmerged same-author branch was an earlier active-PR-limit cleanup submission for the same change.
superseded: fix(gateway): expose idempotencyKey on chat.history __openclaw envelope (#79844) #80044 - This earlier closed unmerged PR attempted the same Gateway idempotency metadata fix and is no longer the viable landing path.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P1] Two broader CI shards were still in progress during this read-only review, so merge should wait for the normal required checks to finish.

Maintainer options:

Decide the mitigation before merge
Land the focused Gateway projection fix after required CI and maintainer review, then let chat.history: surface idempotencyKey on hydrated user messages for client-side correlation #79844 close from the merged fix.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No automated repair lane is needed; the remaining action is normal maintainer review and completion of required CI.

Security
Cleared: The diff only changes Gateway metadata projection and tests; it does not touch dependencies, workflows, secrets, install scripts, package metadata, or other supply-chain surfaces.

Review details

Best possible solution:

Land the focused Gateway projection fix after required CI and maintainer review, then let #79844 close from the merged fix.

Do we have a high-confidence way to reproduce the issue?

Yes. From source, current main persists the user-turn idempotencyKey but omits it from __openclaw in hydrated history, inline history appends, and session.message metadata; the PR body also supplies after-fix loopback Gateway terminal proof.

Is this the best way to solve the issue?

Yes. The PR fixes the owner boundary by surfacing already-persisted Gateway metadata through the existing projection envelope instead of adding client heuristics, config, or a storage migration.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 08d15ec32db3.

Label changes

Label justifications:

P2: The PR fixes a bounded Gateway/Web UI session-correlation bug with limited blast radius.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal proof from a real loopback Gateway chat.send -> agent.wait -> chat.history flow returning __openclaw.idempotencyKey after the fix.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal proof from a real loopback Gateway chat.send -> agent.wait -> chat.history flow returning __openclaw.idempotencyKey after the fix.

Evidence reviewed

PR surface:

Source +87, Tests +80. Total +167 across 7 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	4	87	0	+87
Tests	3	83	3	+80
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	7	170	3	+167

What I checked:

Repository policy read: Root and gateway-scoped AGENTS.md were read; the gateway hot-path and review-depth guidance applies to this PR. (AGENTS.md:1, 08d15ec32db3)
Current main persists the requested key: Current main stores params.idempotencyKey on persisted user-turn transcript messages, so the data exists before history projection. (src/sessions/user-turn-transcript.ts:245, 08d15ec32db3)
Current main omits it from hydrated metadata: Current main attaches id, timestamp, and seq in parsedSessionEntryToMessage but does not copy message.idempotencyKey into __openclaw. (src/gateway/session-utils.fs.ts:841, 08d15ec32db3)
PR head fixes hydrated history: The PR head reads the transcript message idempotency key and includes it in metadata passed to attachOpenClawTranscriptMeta. (src/gateway/session-utils.fs.ts:902, 14643e647e38)
PR head fixes live history appends: The inline history append path now carries a non-empty message idempotency key into __openclaw alongside id and seq. (src/gateway/session-history-state.ts:270, 14643e647e38)
PR head fixes session.message broadcasts: The session-message broadcaster now reads the raw transcript update message idempotency key before display projection and includes it in __openclaw. (src/gateway/server-session-events.ts:184, 14643e647e38)

Likely related people:

giodl73-repo: Current-line blame for the affected hydration, live-history append, session-message broadcast, and user-turn persistence lines points to the merged Gateway transcript/state refactor in pull request Doctor: expose state integrity findings #95979. (role: recent central Gateway transcript contributor; confidence: high; commits: cb4244fe15e3; files: src/gateway/session-utils.fs.ts, src/gateway/session-history-state.ts, src/gateway/server-session-events.ts)
galiniliev: Recent merged pagination work changed session-utils.fs.ts, session-transcript-readers.ts, and server-methods/chat.ts, including the same bounded history and oversized placeholder surfaces this PR updates. (role: recent adjacent chat-history contributor; confidence: high; commits: dc575d148a7c; files: src/gateway/session-utils.fs.ts, src/gateway/session-transcript-readers.ts, src/gateway/server-methods/chat.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

wyf027 · 2026-06-28T11:36:53Z

Addressed the ClawSweeper follow-up on the latest head (af0b13a892):

Fixed oversized transcript idempotencyKey extraction for the production field order where large content appears before idempotencyKey.
Updated the oversized regression fixture to match that production order.
Updated the PR body with real loopback Gateway proof showing chat.send -> agent.wait -> chat.history returning __openclaw.idempotencyKey.

Verification run after the patch:

node_modules/.bin/vitest run --config test/vitest/vitest.gateway.config.ts src/gateway/session-utils.fs.test.ts --reporter=dot
node_modules/.bin/oxfmt --check --threads=1 src/gateway/session-utils.fs.ts src/gateway/session-utils.fs.test.ts
git diff --check

@clawsweeper re-review

clawsweeper · 2026-06-28T11:36:55Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

…ry-idempotency # Conflicts: # src/gateway/session-utils.fs.ts

wyf027 · 2026-06-28T11:44:54Z

Updated again after merging current upstream/main to clear the PR's CONFLICTING merge state.

Latest head: c5472e7cfc

Additional verification after conflict resolution:

node_modules/.bin/vitest run --config test/vitest/vitest.gateway.config.ts src/gateway/session-utils.fs.test.ts --reporter=dot
node_modules/.bin/oxfmt --check --threads=1 src/gateway/session-utils.fs.ts src/gateway/session-utils.fs.test.ts
git diff --check

@clawsweeper re-review

clawsweeper · 2026-06-28T11:44:57Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

wyf027 · 2026-06-28T11:55:40Z

Fixed the CI build root cause from the previous head.

upstream/main had moved the bounded transcript JSON helpers into src/gateway/session-transcript-json.ts; the conflict resolution briefly re-declared those imported helpers in src/gateway/session-utils.fs.ts, which caused SyntaxError: Identifier 'extractJsonStringFieldPrefix' has already been declared in build/startup paths.

Latest head 14643e647e now reuses the shared imported helpers and keeps only the oversized suffix scanner local to session-utils.fs.ts.

Local verification after this fix:

node_modules/.bin/vitest run --config test/vitest/vitest.gateway.config.ts src/gateway/session-utils.fs.test.ts --reporter=dot
node_modules/.bin/oxfmt --check --threads=1 src/gateway/session-utils.fs.ts src/gateway/session-utils.fs.test.ts
git diff --check

@clawsweeper re-review

clawsweeper · 2026-06-28T11:55:43Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

…w#96273) * fix(gateway): surface idempotency key in history metadata * test(gateway): avoid unsafe optional chain * fix(gateway): preserve oversized transcript idempotency keys * fix(gateway): preserve oversized idempotency keys * fix(gateway): reuse transcript json helpers --------- Co-authored-by: 吴杨帆 <[email protected]>

wyf027 and others added 2 commits June 18, 2026 11:13

fix(gateway): surface idempotency key in history metadata

8543f6f

test(gateway): avoid unsafe optional chain

06166c7

wyf027 mentioned this pull request Jun 24, 2026

fix(gateway): expose idempotencyKey in chat history metadata #94438

Closed

openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime size: S labels Jun 24, 2026

chatgpt-codex-connector Bot reviewed Jun 24, 2026

View reviewed changes

Comment thread src/gateway/session-utils.fs.ts

fix(gateway): preserve oversized transcript idempotency keys

6618db5

fix(gateway): preserve oversized idempotency keys

af0b13a

Merge remote-tracking branch 'upstream/main' into pr-96273-chat-histo…

c5472e7

…ry-idempotency # Conflicts: # src/gateway/session-utils.fs.ts

openclaw-barnacle Bot added size: M and removed size: S labels Jun 28, 2026

fix(gateway): reuse transcript json helpers

14643e6

openclaw-barnacle Bot added size: S and removed size: M labels Jun 28, 2026

vincentkoc merged commit f9dddea into openclaw:main Jun 28, 2026
121 of 130 checks passed

github-actions Bot mentioned this pull request Jun 28, 2026

📡 Upstream Digest — 2026-06-28 19:05 UTC curtismercier/openclaw-mods#1168

Open

clawsweeper Bot mentioned this pull request Jun 30, 2026

WebChat UI 会话合并后消息顺序错乱 #98001

Closed

clawsweeper Bot mentioned this pull request Jul 1, 2026

feat(durable): wire agent session runtime coordination #98719

Closed

This was referenced Jul 3, 2026

fix(gateway): preserve text from oversized multimodal history #78730

Closed

[AI-assisted] Add session history family lookup #90239

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(gateway): expose idempotencyKey in chat history metadata#96273

fix(gateway): expose idempotencyKey in chat history metadata#96273
vincentkoc merged 6 commits into
openclaw:mainfrom
wyf027:codex/chat-history-idempotency-key-79844-reopen-20260618

wyf027 commented Jun 24, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

wyf027 commented Jun 24, 2026

Uh oh!

clawsweeper Bot commented Jun 24, 2026 •

edited

Loading

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

wyf027 commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Real behavior proof

What Problem This Solves

Evidence

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

wyf027 commented Jun 24, 2026

Uh oh!

clawsweeper Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wyf027 commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wyf027 commented Jun 24, 2026 •

edited

Loading

clawsweeper Bot commented Jun 24, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading