Skip to content

fix(agents): avoid synthetic tool results during parallel races (#88168)#88588

Merged
steipete merged 1 commit into
openclaw:mainfrom
TurboTheTurtle:fix/parallel-tool-result-race-88168
May 31, 2026
Merged

fix(agents): avoid synthetic tool results during parallel races (#88168)#88588
steipete merged 1 commit into
openclaw:mainfrom
TurboTheTurtle:fix/parallel-tool-result-race-88168

Conversation

@TurboTheTurtle

@TurboTheTurtle TurboTheTurtle commented May 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Stops the session tool-result guard from inserting synthetic missing-result entries at the moment a newer assistant tool-call turn arrives while older tool results may still be in flight.
  • Lets transcript repair move late real tool results back next to their matching assistant tool calls before synthesizing placeholders.
  • Preserves missing-result repair for genuinely absent tool results and duplicate/orphan cleanup.
  • Intentionally out of scope: changing Anthropic streaming/tool execution ordering outside the session transcript guard and repair layer.

Linked context

Closes #88168

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: Parallel tool-call batches should not persist [openclaw] missing tool result... placeholders before real in-flight tool results have a chance to land and be repaired into provider-valid order.

  • Real environment tested: Local OpenClaw source checkout at /private/tmp/openclaw-88168 on macOS, branch fix/parallel-tool-result-race-88168, rebased onto f7a1d3f3f6, running the repo's real SessionManager guard path, transcript repair path, Vitest, and tsgo test runners against the changed code.

  • Exact steps or command run after this patch: timeout 180 node scripts/test-projects.mjs src/agents/session-tool-result-guard.test.ts src/agents/session-transcript-repair.test.ts; node --import tsx -e '<inline script using SessionManager.inMemory(), installSessionToolResultGuard(), and repairToolUseResultPairing() with late real parallel tool results>'.

  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output): After rebasing to f7a1d3f3f6, the focused test run passed 1 Vitest shard with 2 passed / 73 tests. git diff --check exited 0. Type checking exited 0 for timeout 240 node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.src.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/test-src-88168-rebased-f7a1.tsbuildinfo.

    Redacted local runtime/session proof from the real SessionManager guard and transcript repair paths after rebasing to f7a1d3f3f6:

    {
      "proof": "SessionManager guard plus transcript repair local runtime path",
      "base": "f7a1d3f3",
      "pendingAfterLateResults": [],
      "repairAddedSyntheticCount": 0,
      "repairMovedRealResults": true,
      "rawContainsMissingPlaceholder": false,
      "repairedContainsMissingPlaceholder": false,
      "repaired": [
        { "role": "assistant", "toolCallId": null, "isError": null, "text": "no missing placeholder" },
        { "role": "toolResult", "toolCallId": "call_read", "isError": false, "text": "no missing placeholder" },
        { "role": "assistant", "toolCallId": null, "isError": null, "text": "no missing placeholder" },
        { "role": "toolResult", "toolCallId": "call_exec", "isError": false, "text": "no missing placeholder" }
      ]
    }
  • Observed result after fix: The SessionManager guard accepts two assistant tool-call turns followed by late real results without persisting synthetic missing-result text; transcript repair then moves the real call_read result beside its assistant turn and adds zero synthetic results.

  • What was not tested: A live Anthropic Claude session with real parallel exec/read calls was not run from this workstation.

  • Proof limitations or environment constraints: This uses a local SessionManager runtime invocation plus repository tests rather than live provider proof; the original production session JSONL and Anthropic credentials are not available here. The later .jsonl.lock field report on Synthetic 'missing tool result' entries injected for parallel tool calls on Anthropic Claude, despite real results being produced #88168 is treated as separate follow-up scope unless maintainers want it folded into this PR.

  • Before evidence (optional but encouraged): Issue Synthetic 'missing tool result' entries injected for parallel tool calls on Anthropic Claude, despite real results being produced #88168 documents live sessions where real parallel tool side effects completed but persisted transcript history contained synthetic [openclaw] missing tool result in session history; inserted synthetic error result for transcript repair. entries instead of the real outputs.

Tests and validation

  • timeout 180 node scripts/test-projects.mjs src/agents/session-tool-result-guard.test.ts src/agents/session-transcript-repair.test.ts
  • timeout 240 node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.src.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/test-src-88168-rebased-f7a1.tsbuildinfo
  • node --import tsx -e '<inline script using SessionManager.inMemory(), installSessionToolResultGuard(), and repairToolUseResultPairing() with late real parallel tool results>'
  • git diff --check

Regression coverage was added for the guard not synthesizing at the new-assistant boundary and for transcript repair moving late real results ahead of newer assistant tool calls.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Parallel tool-call transcripts should retain real results instead of premature synthetic error placeholders.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

Delaying synthetic missing-result insertion could allow a temporarily invalid raw transcript shape until repair runs.

How is that risk mitigated?

The existing non-tool-result flush path still synthesizes genuinely missing results, and transcript repair now deterministically reorders late real results or synthesizes missing siblings before replay.

Current review state

What is the next action?

Await ClawSweeper re-review and fresh CI after the rebase/proof update.

What is still waiting on author, maintainer, CI, or external proof?

CI, ClawSweeper, and maintainer review. No known author-side blocker after this proof update.

Which bot or reviewer comments were addressed?

Addressed ClawSweeper's request for redacted runtime/session proof showing late real tool results repaired without synthetic placeholders, then refreshed that proof after rebasing to f7a1d3f3f6.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 31, 2026
@clawsweeper

clawsweeper Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 31, 2026, 6:52 AM ET / 10:52 UTC.

Summary
The PR changes session tool-result persistence and transcript repair so late real tool results can be retained and reordered instead of being replaced by premature synthetic placeholders, with focused regression tests.

PR surface: Source +44, Tests +71. Total +115 across 4 files.

Reproducibility: no. I did not live-reproduce current main. Source inspection shows the current guard still flushes pending tool calls before a newer assistant tool-call turn, and the PR body supplies after-fix local SessionManager/repair proof for that interleaving.

Review metrics: 1 noteworthy metric.

  • Runtime Transcript Paths: 2 changed. The diff changes both the persistence-time guard and replay-time repair, so maintainers should review raw transcript and replay behavior together before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Maintainer should decide whether the .jsonl.lock field report is separate follow-up scope before closing the linked issue.

Risk before merge

  • [P1] The patch intentionally allows a temporarily invalid raw transcript ordering until replay/repair runs, so maintainers should be comfortable that every provider replay path using this policy repairs before sending history.
  • [P1] The supplied proof covers local SessionManager and transcript-repair runtime paths plus CI, but not a live Anthropic provider session with real parallel tool execution.
  • [P1] The linked issue includes a later .jsonl.lock field report on the synthetic-write path; this PR treats that as separate scope, and no separate canonical issue was found in the targeted search.

Maintainer options:

  1. Merge With Explicit Follow-Up Scope (recommended)
    Maintainers can land this focused fix if they explicitly track or defer the .jsonl.lock report separately before closing the linked issue.
  2. Fold Lock Proof Into This PR
    Require the author or a maintainer to prove that the synthetic-write path cannot keep the session lock held before merging this branch.
  3. Pause If The Lock Report Is Same Root
    If maintainers decide the lock leak is part of the same root cause, pause this PR until the lock-release behavior is fixed or covered by a separate viable PR.

Next step before merge

  • No automated repair is indicated; maintainer review should decide merge readiness and whether the linked lock report is tracked separately before issue closure.

Security
Cleared: The diff only changes agent session transcript code and colocated tests; it does not add dependencies, CI, permissions, secrets handling, or new code-execution supply-chain surface.

Review details

Best possible solution:

Land the focused transcript fix after maintainers confirm the lock report is separate follow-up scope, then close the linked issue only for the synthetic placeholder race this PR actually resolves.

Do we have a high-confidence way to reproduce the issue?

No, I did not live-reproduce current main. Source inspection shows the current guard still flushes pending tool calls before a newer assistant tool-call turn, and the PR body supplies after-fix local SessionManager/repair proof for that interleaving.

Is this the best way to solve the issue?

Yes for the central placeholder race: the guard boundary plus repair fallback is a narrow fix in the implicated paths. The safer merge state is to make the .jsonl.lock report an explicit follow-up or add lock proof here before closing the linked issue.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 1e08af453a06.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted after-fix local runtime output from the real SessionManager guard and transcript repair paths, plus focused test/typecheck results; live Anthropic proof remains a maintainer risk, not a missing contributor-proof gate.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P1: The PR targets a current agent workflow that can replace real parallel tool outputs with synthetic placeholders and corrupt session replay.
  • merge-risk: 🚨 session-state: Merging changes when synthetic tool-result placeholders are written and how late real results are repaired in persisted session history.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): The PR body includes redacted after-fix local runtime output from the real SessionManager guard and transcript repair paths, plus focused test/typecheck results; live Anthropic proof remains a maintainer risk, not a missing contributor-proof gate.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted after-fix local runtime output from the real SessionManager guard and transcript repair paths, plus focused test/typecheck results; live Anthropic proof remains a maintainer risk, not a missing contributor-proof gate.
Evidence reviewed

PR surface:

Source +44, Tests +71. Total +115 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 56 12 +44
Tests 2 71 0 +71
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 127 12 +115

Acceptance criteria:

  • [P1] Review CI for head 59d2447, especially agent runtime shards, build-artifacts, check-lint, check-prod-types, check-test-types, security-fast, and Real behavior proof.
  • [P1] If credentials are available, run a live Anthropic/webchat parallel exec/read or process-poll session and inspect the session JSONL for real toolResult entries without synthetic placeholders.
  • [P1] Before closing the linked issue, either open/link a separate issue for the .jsonl.lock report or add proof that this PR covers that lock-release behavior.

What I checked:

  • Repository policy read: Root AGENTS.md and scoped src/agents/AGENTS.md were read fully; the root review policy treats auth/session state and replay behavior as compatibility-sensitive, and the scoped file calls out agent test architecture and proof expectations. (AGENTS.md:1, 1e08af453a06)
  • Current main still has the early flush boundary: Current main flushes pending tool results whenever a new assistant tool-call turn arrives while pending IDs exist, which is the source-side condition described by the linked issue. (src/agents/session-tool-result-guard.ts:748, 1e08af453a06)
  • PR narrows synthetic flushing at the new assistant boundary: The PR head only calls the new-tool-call flush when synthetic results are disabled, leaving strict-provider synthetic repair to later replay/repair instead of this racing append boundary. (src/agents/session-tool-result-guard.ts:748, 59d2447234e5)
  • PR teaches repair to recover late real results: The PR head adds a later-result lookup and uses it before synthesizing a missing tool result, so a real result displaced behind a newer assistant tool-call turn can be moved back into provider-valid order. (src/agents/session-transcript-repair.ts:448, 59d2447234e5)
  • Regression tests cover both changed paths: The PR adds one guard test for not synthesizing before a newer assistant tool-call turn and one repair test for moving late real results ahead of newer assistant calls without adding synthetic placeholders. (src/agents/session-tool-result-guard.test.ts:373, 59d2447234e5)
  • Related issue has a later lock-scope report: The linked issue comments include a later field report that the same synthetic write path may leave a session .jsonl.lock held; no separate canonical issue was found by targeted GitHub search for that lock report.

Likely related people:

  • steipete: Peter Steinberger authored the original session tool-result guard commit and recent current-main guard/repair code is attributed to adjacent commits in this path. (role: original guard author and recent area contributor; confidence: high; commits: f5d5661adfec, 8f2e520abb50; files: src/agents/session-tool-result-guard.ts, src/agents/session-tool-result-state.ts, src/agents/session-transcript-repair.ts)
  • vignesh07: Vignesh Natarajan authored a prior fix that avoided synthetic tool-result writes during idle-timeout cleanup in the same guard surface. (role: adjacent synthetic-result cleanup contributor; confidence: medium; commits: 4daaea119041; files: src/agents/session-tool-result-guard.ts)
  • shakkernerd: Shakker authored the prior change that flushes pending tool results when assistant tool-call sanitization drops a message, adjacent to the pending-map behavior under review. (role: adjacent guard behavior contributor; confidence: medium; commits: befa421a57cf; files: src/agents/session-tool-result-guard.ts, src/agents/session-tool-result-guard.test.ts)
  • jiangnan: jiangnan authored a prior change clearing pending tool-call state on interruption, relevant to the same pending-map lifecycle. (role: adjacent pending-state contributor; confidence: medium; commits: 11aa18b52545; files: src/agents/session-tool-result-guard.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 31, 2026
@TurboTheTurtle
TurboTheTurtle force-pushed the fix/parallel-tool-result-race-88168 branch from 3075901 to bf66381 Compare May 31, 2026 10:26

Copy link
Copy Markdown
Contributor Author

Updated the PR body with redacted local SessionManager/transcript-repair proof after rebasing to 4b1e5b79435c.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@TurboTheTurtle
TurboTheTurtle force-pushed the fix/parallel-tool-result-race-88168 branch from bf66381 to 19264bc Compare May 31, 2026 10:39
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 31, 2026
@TurboTheTurtle
TurboTheTurtle force-pushed the fix/parallel-tool-result-race-88168 branch from 19264bc to 59d2447 Compare May 31, 2026 10:45
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased again and refreshed the PR body/proof on head 59d2447234e5f00d314a27fd091f96e329600afc. Raw Pulls API now verifies maintainer_can_modify: true.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels May 31, 2026
@steipete steipete self-assigned this May 31, 2026

@steipete steipete left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review: approved for the focused #88168 placeholder race fix.

What I checked:

  • Diff is scoped to src/agents/session-tool-result-guard.ts, src/agents/session-transcript-repair.ts, and focused tests.
  • Root cause matches current main: the guard flushes pending tool results at the new assistant tool-call boundary, which can synthesize placeholders while real parallel results are still racing.
  • Fix shape is right for this surface: stop synthesizing at that race boundary when synthetic results are enabled, and let transcript repair move late real results before inserting missing-result placeholders.
  • Regression coverage covers both changed paths: guard persistence behavior and replay repair ordering.
  • ClawSweeper re-review is proof: sufficient / ready for maintainer look at head 59d2447234e5f00d314a27fd091f96e329600afc.

Local proof:

  • git diff --check origin/main...HEAD
  • node scripts/test-projects.mjs src/agents/session-tool-result-guard.test.ts src/agents/session-transcript-repair.test.ts -> 2 files, 73 tests passed.

CI/proof reviewed:

  • Head SHA: 59d2447234e5f00d314a27fd091f96e329600afc
  • Live merge state: MERGEABLE / CLEAN, no pending checks.
  • CI run 26710452128 passed the relevant check, lint, type, build, security, and agent/runtime shards.
  • Real behavior proof check completed successfully at 2026-05-31T10:53:06Z.

Known proof gap:

  • No live Anthropic parallel tool session was run by me; contributor proof and local focused runtime/tests are sufficient for this small session-state fix.
  • The later .jsonl.lock field report from #88168 is tracked separately in #88647, so this PR closes the synthetic placeholder race only.

@steipete
steipete merged commit 44c65de into openclaw:main May 31, 2026
167 of 170 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Synthetic 'missing tool result' entries injected for parallel tool calls on Anthropic Claude, despite real results being produced

2 participants