Skip to content

fix: skip Responses item id replay without store#89128

Closed
snowzlm wants to merge 3 commits into
openclaw:mainfrom
snowzlm:fix/responses-store-false-replay
Closed

fix: skip Responses item id replay without store#89128
snowzlm wants to merge 3 commits into
openclaw:mainfrom
snowzlm:fix/responses-store-false-replay

Conversation

@snowzlm

@snowzlm snowzlm commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

What problem does this PR solve?

  • Prevents OpenClaw from replaying prior OpenAI Responses item ids when the effective request is non-persistent (store=false or store is stripped because the custom provider declares compat.supportsStore=false).
  • Fixes the embedded-reply failure mode where custom OpenAI-compatible Responses backends reject reused item ids with errors like 404 Item with id ... not found and Items are not persisted when store is set to false.
  • Keeps existing replay behavior for store-enabled OpenAI Responses requests so the change stays scoped to non-persistent routes.

Why does this matter now?

  • Custom Responses-compatible providers are already used in real OpenClaw deployments, and this bug turns follow-up/tool-turn replies into Something went wrong while processing your request failures.

What is the intended outcome?

  • Non-persistent Responses routes stop sending stale replay item ids.
  • Persistent/store-enabled Responses routes continue replaying item ids as before.

What is intentionally out of scope?

  • This PR does not change the UI-side startup: false assertion failure currently red in upstream CI.
  • This PR does not attempt to fix the unrelated long-running Vitest shard timeouts in infra-file-safety and startup-health-runtime.

What does success look like?

  • OpenClaw omits replay item ids for store=false / supportsStore=false Responses requests.
  • Focused transport and wrapper regression coverage passes.
  • Typecheck/lint/build checks directly touched by this change stay green.

What should reviewers focus on?

  • The replayResponsesItemIds gating in src/llm/providers/stream-wrappers/openai.ts.
  • The transport-side guard in src/agents/openai-transport-stream.ts.
  • The new regression tests covering both store-disabled and store-enabled behavior.

Linked context

Which issue does this close?

Closes #

Which issues, PRs, or discussions are related?

Related #

Was this requested by a maintainer or owner?

  • Requested by the repo owner's downstream fork workflow while preparing an upstream fix.

Real behavior proof (required for external PRs)

  • Behavior or issue addressed:
    • A custom OpenAI-compatible Responses route reused prior response item ids even though the effective request was non-persistent, causing backend errors (404 Item with id ... not found, Items are not persisted when store is set to false) and embedded reply failure.
  • Real environment tested:
    • Real custom Responses-compatible backend: aitokenapi/gpt-5.5 at catcats.net.
    • Real branch runtime payload generation from this PR via buildOpenAIResponsesParams(...) in the patched source tree.
  • Exact steps or command run after this patch:
    • pnpm exec tsx projects/openclaw-upstream-fix/reports/pr-89128-real-proof/capture-real-backend-proof.ts
  • Evidence after fix (redacted):
    • Request summary:
      • store field present: false
      • reasoning item id present: false
      • assistant message id present: false
      • function_call id present: false
      • function_call call_id: call_abc
      • reasoning encrypted_content present: false
    • Response summary:
      • HTTP status: 200
      • response status: completed
      • previous_response_id: null
      • output_count: 1
    • Redacted response preview:
      • {"status":"completed","previous_response_id":null,...}
  • Terminal output after fix (redacted):
provider=aitokenapi
model=gpt-5.5
http_status=200
response_status=completed
store_field_present=false
reasoning_id_present=false
assistant_message_id_present=false
function_call_id_present=false
function_call_call_id=call_abc
reasoning_encrypted_content_present=false
previous_response_id=null
output_count=1
  • Observed result after fix:
    • The real backend accepted the non-persistent follow-up payload and completed successfully without replaying stale Responses item ids.
  • What was not tested:
    • I did not attach raw credential-bearing upstream transport logs or run the same proof against every custom Responses-compatible provider configured locally.
    • I also did not run a separately installed long-running OpenClaw gateway outside this source checkout as part of this PR branch.
  • Local proof artifacts retained for audit:
    • projects/openclaw-upstream-fix/reports/pr-89128-real-proof/real-backend-proof.md
    • projects/openclaw-upstream-fix/reports/pr-89128-real-proof/real-backend-proof.json
    • projects/openclaw-upstream-fix/reports/pr-89128-real-proof/real-backend-proof-terminal.txt
  • Proof limitations or environment constraints:
    • This proof uses a redacted local audit artifact rather than attaching raw credential-bearing transport logs.
    • It exercises the real backend with the patched request builder/path and confirms the no-stale-id request shape plus successful completion.

Tests and validation

Which commands did you run?

  • pnpm test src/agents/openai-transport-stream.test.ts src/agents/embedded-agent-runner-extraparams.test.ts
  • gh workflow run ci.yml --repo snowzlm/openclaw --ref fix/responses-store-false-replay -f target_ref=fix/responses-store-false-replay -f include_android=false

What regression coverage was added or updated?

  • Added wrapper-level coverage proving replayResponsesItemIds=false when a custom Responses provider strips store support.
  • Added transport-level coverage proving reasoning/message/function_call item ids are omitted for non-persistent custom Responses routes.
  • Preserved existing store-enabled replay coverage to avoid regressing native OpenAI Responses behavior.

What failed before this fix, if known?

  • Custom Responses backends that do not persist items rejected replayed item ids on follow-up turns, breaking embedded replies.

If no test was added, why not?

  • Tests were added.

Risk checklist

Did user-visible behavior change? (Yes/No)

  • Yes

Did config, environment, or migration behavior change? (Yes/No)

  • No

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

  • No

What is the highest-risk area?

  • Accidentally disabling replay item ids for store-enabled OpenAI Responses requests that still rely on persisted item history.

How is that risk mitigated?

  • The guard only disables replay when explicitStore === false or shouldStripStore === true.
  • Existing store-enabled behavior is preserved and explicitly covered by regression tests.

Current review state

What is the next action?

  • Review the replay gating logic and confirm the unrelated red CI jobs can be ignored or handled separately.

What is still waiting on author, maintainer, CI, or external proof?

  • No author changes are pending for the Responses replay fix itself.
  • Maintainer review is needed on whether to merge with the three unrelated red CI jobs still present in upstream/fork runs.

Which bot or reviewer comments were addressed?

  • Addressed the initial check-test-types regression from this branch by relaxing the test model typing in follow-up commit 84a9cce5c7.

Issue

Fixes #89330

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 1, 2026
@clawsweeper

clawsweeper Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 2, 2026, 1:22 AM ET / 05:22 UTC.

Summary
The branch changes OpenAI Responses replay gating so store=false or compat.supportsStore=false routes omit prior Responses item IDs while store-enabled OpenAI routes continue replaying them, with focused regression tests.

PR surface: Source +3, Tests +137. Total +140 across 4 files.

Reproducibility: yes. Current main has a clear source path where compat.supportsStore=false sets shouldStripStore but the replay decision ignores that bit, and the PR body supplies redacted live-backend after-fix proof.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] I did not rerun the focused tests or live backend proof in this read-only pass; required CI or maintainer-run validation should still gate merge.

Maintainer options:

  1. Decide the mitigation before merge
    Land the focused replay-policy guard with the added regression tests once normal required checks and maintainer review are satisfied.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No repair lane is needed because the branch already contains the focused fix, regression coverage, and sufficient real behavior proof; the remaining action is maintainer review and required checks.

Security
Cleared: The diff only changes provider payload gating and tests; it adds no dependencies, workflows, scripts, permissions, or secret-handling surface.

Review details

Best possible solution:

Land the focused replay-policy guard with the added regression tests once normal required checks and maintainer review are satisfied.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main has a clear source path where compat.supportsStore=false sets shouldStripStore but the replay decision ignores that bit, and the PR body supplies redacted live-backend after-fix proof.

Is this the best way to solve the issue?

Yes. The patch fixes the replay decision at the policy boundary used by both the wrapper and transport builder, while tests preserve store-enabled OpenAI replay behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 6883351085c3.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted terminal output from a real custom Responses-compatible backend showing replay IDs omitted and the request completing with HTTP 200 after the patch.

Label justifications:

  • P2: The PR fixes a real provider/runtime bug that can break follow-up replies for custom OpenAI Responses-compatible routes, with limited blast radius.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes redacted terminal output from a real custom Responses-compatible backend showing replay IDs omitted and the request completing with HTTP 200 after the patch.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted terminal output from a real custom Responses-compatible backend showing replay IDs omitted and the request completing with HTTP 200 after the patch.
Evidence reviewed

PR surface:

Source +3, Tests +137. Total +140 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 7 4 +3
Tests 2 137 0 +137
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 144 4 +140

What I checked:

  • Root and scoped policy read: The full root AGENTS.md and scoped src/agents/AGENTS.md were read and applied; their review policy requires dependency/Codex contract checks, sibling-path review, real proof, and best-fix assessment for this provider/runtime change. (AGENTS.md:1, 6883351085c3)
  • Current-main failure boundary: Current main defaults replay from payloadPolicy.explicitStore !== false; when supportsStore=false, the policy can strip store without setting explicitStore=false, so prior item IDs can still be replayed. (src/agents/openai-transport-stream.ts:2195, 6883351085c3)
  • Store-stripping policy source: resolveOpenAIResponsesPayloadPolicy sets shouldStripStore for Responses models whose compat declares supportsStore=false, which is the policy bit the PR adds to the replay decision. (src/agents/openai-responses-payload-policy.ts:353, 6883351085c3)
  • Runtime wrapper entry point: Embedded runs apply the OpenAI Responses context-management wrapper in the post-plugin stream-wrapper path, so fixing the wrapper covers the normal custom-provider route as well as the transport helper. (src/agents/embedded-agent-runner/extra-params.ts:849, 6883351085c3)
  • Focused PR diff: The runtime diff changes only the transport replay default and wrapper replay override, and adds regression coverage for custom supportsStore=false plus existing store-enabled replay behavior. (src/llm/providers/stream-wrappers/openai.ts:399, 0d6de5ec36ca)
  • OpenAI SDK contract: The locked OpenAI SDK is 6.39.1; its Responses types expose store, previous_response_id, replayable output item IDs, and note reasoning.encrypted_content for stateless store=false conversations, matching the request-shape boundary this PR changes. (pnpm-lock.yaml:155, 6883351085c3)

Likely related people:

  • Vincent Koc: Current blame for the central replay-policy and wrapper lines points to this recent OpenAI Responses transport/payload policy history. (role: recent area contributor; confidence: medium; commits: 2f92fddef088; files: src/agents/openai-transport-stream.ts, src/llm/providers/stream-wrappers/openai.ts, src/agents/openai-responses-payload-policy.ts)
  • Peter Steinberger: Recent history shows a live model inference edge-case fix touched the OpenAI Responses transport and nearby tests shortly before this PR. (role: recent adjacent contributor; confidence: medium; commits: 9ead0ae9219e; files: src/agents/openai-transport-stream.ts, src/agents/openai-transport-stream.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jun 1, 2026
@snowzlm

snowzlm commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Added real backend proof from a custom Responses-compatible route.

Redacted proof summary:

  • backend: aitokenapi/gpt-5.5 @ catcats.net
  • request: store field present=false, reasoning item id present=false, assistant message id present=false, function_call id present=false, function_call call_id=call_abc, reasoning encrypted_content present=false
  • response: HTTP 200, status=completed, previous_response_id=null, output_count=1

The PR body now includes the proof details and retained local audit artifact paths.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@openclaw-barnacle openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 1, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 1, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label Jun 1, 2026
@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 1, 2026
@openclaw-barnacle openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 1, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 1, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 1, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 1, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 2, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 2, 2026
@snowzlm

snowzlm commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Opened and linked tracking issue: #89330

@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 2, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 2, 2026
@steipete

steipete commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Thanks @snowzlm. This fix is now already present on current main via 03dec8bb3a00209780d3140ec3f10ad0eb069030 (fix(openai): avoid replay ids when Responses store is disabled).

That landed commit covers the same non-persistent Responses replay gate and includes the wrapper/transport regressions plus live OpenAI Responses gpt-5.5 proof, so this PR is superseded rather than rejected. Closing to keep the queue accurate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: non-persistent Responses routes replay stale item ids

2 participants