Skip to content

fix(guard): pair-atomic tool_use/tool_result commit — prevent orphaned tool_use in JSONL#20980

Closed
amabito wants to merge 5 commits into
openclaw:mainfrom
amabito:fix/tool-result-guard-pair-atomic
Closed

fix(guard): pair-atomic tool_use/tool_result commit — prevent orphaned tool_use in JSONL#20980
amabito wants to merge 5 commits into
openclaw:mainfrom
amabito:fix/tool-result-guard-pair-atomic

Conversation

@amabito

@amabito amabito commented Feb 19, 2026

Copy link
Copy Markdown

Summary

  • installSessionToolResultGuard now holds assistant messages that contain tool_use blocks in a local pair buffer, committing the pair to JSONL only after all matching tool_result IDs are received.
  • flushPendingToolResults() defaults to discard mode: on any interruption the incomplete pair is dropped without being written, so no orphaned tool_use block ever reaches JSONL.
  • Legacy OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic preserves the previous synthesize-and-commit behavior.

Why

Tool-use aborts (rate-limit, provider failover, user cancel) previously left the session JSONL in an inconsistent state: an assistant tool_use block without a corresponding tool_result. Strict providers (Anthropic, Cloud Code Assist, MiniMax) reject any subsequent API call that includes an orphaned tool_use in its history, causing a hard 400 error that survives across sessions until the JSON file is manually repaired.

Changes

session-tool-result-guard.ts

  • Added toolPairBuffer: AgentMessage[] alongside the existing pending map.
  • guardedAppend: assistant messages with tool_use are buffered rather than written to JSONL; each incoming tool_result advances the buffer; when pending.size === 0, commitToolPairBuffer() writes the whole pair atomically.
  • validatePairIntegrity() guards the commit: if a buffer somehow arrives with mismatched IDs it is discarded instead of committed.
  • flushPendingToolResults: new env-var-driven abort mode (see table below).

Why discard on hook suppression?
In the hook-suppressed path we intentionally discard the in-flight buffer instead of attempting a commit, because a suppressed tool_result would otherwise allow a lone tool_use to persist and corrupt the transcript. This keeps the pair-atomic contract strict. For environments that relied on legacy behavior, OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic preserves the prior "synthesize-and-commit" semantics (only when allowSyntheticToolResults is enabled).

attempt.ts

  • Added FAILOVER COVERAGE comment to the finally block documenting this as the single flush point for all exit paths (abort, rate-limit, provider failover).

Abort mode

OPENCLAW_TOOL_GUARD_ABORT_MODE Behavior
unset / discard (new default) buffer discarded; structured log emitted
synthetic synthetic error tool_result synthesized and committed (legacy)

Tests

New — session-tool-result-guard.atomic-commit.test.ts

5 cases:

  • Case 1: abort during tool execution → 0 messages, no orphan
  • Case 2: failover during tool execution → 0 messages, no orphan
  • Case 3: rate-limit interruption → user message preserved, no orphan
  • Case 4: normal pair → [assistant, toolResult] committed in order
  • Case 5: first pair committed, second aborted → only first pair in JSONL

Updated — pi-embedded-runner.guard.waitforidle-before-flush.test.ts

  • Intermediate assertion: nothing in JSONL while pair is in flight (was incorrect: showed assistant prematurely).
  • Added timeout / discard-mode test (expects 0 entries).
  • Added timeout / synthetic-mode test (expects [assistant, toolResult] with synthetic error).
  • Env var management via beforeEach/afterEach.

Greptile Summary

Implements pair-atomic tool_use/tool_result commit to prevent orphaned tool_use blocks in session JSONL when runs are interrupted (abort, rate-limit, provider failover). The guard now buffers assistant messages containing tool_use blocks and only commits the complete pair to JSONL after all matching tool_result IDs arrive. On interruption, the incomplete pair is discarded by default (new discard mode), ensuring no corrupt state reaches JSONL. Legacy synthetic mode preserves the previous behavior via OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic.

Key improvements:

  • Introduced toolPairBuffer to hold assistant messages with tool_use until their results complete
  • Added validatePairIntegrity() to ensure all tool_use IDs have matching tool_result IDs before commit
  • New default discard mode drops incomplete pairs on any interruption (abort, failover, rate-limit, hook suppression)
  • Comprehensive test coverage for all abort scenarios and pair commit edge cases
  • Added dropOrphanedToolPairs() utility for session repair when loading corrupted transcripts

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The implementation is well-designed with strong safeguards: pair-atomic commit ensures JSONL consistency, comprehensive test coverage validates all abort scenarios (5 new tests covering abort/failover/rate-limit/normal/sequential cases), backward compatibility is maintained via environment variable, and the discard-by-default behavior is conservative and safe. The logic is clean, well-commented, and follows the repository's coding standards.
  • No files require special attention

Last reviewed commit: c34ea89

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: L labels Feb 19, 2026
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from c34ea89 to c9b044b Compare February 19, 2026 14:27
@openclaw-barnacle openclaw-barnacle Bot added channel: discord Channel integration: discord cli CLI command changes commands Command implementations channel: feishu Channel integration: feishu labels Feb 19, 2026
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from c9b044b to 96d8cb8 Compare February 19, 2026 14:32
@openclaw-barnacle openclaw-barnacle Bot removed the channel: feishu Channel integration: feishu label Feb 19, 2026

@nikolasdehor nikolasdehor left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thorough review of the full diff. This is a well-designed fix for a real problem — orphaned tool_use blocks in session JSONL causing hard 400 errors on Anthropic/Cloud Code Assist that survive across sessions.

The pair-atomic approach is the right call: buffer the assistant tool_use message, accumulate matching tool_result IDs via the pending map, commit the whole pair only when pending.size === 0. The validatePairIntegrity() guard at commit time is a good safety net against edge cases where IDs somehow mismatch.

Key things I verified:

  1. All exit paths flush correctly. The finally block in attempt.ts is the single flush point for abort/rate-limit/failover. run.ts calls advanceAuthProfile() only after runEmbeddedAttempt() returns, so no orphaned state leaks into the next provider attempt.

  2. Hook suppression is handled. When applyBeforeWriteHook suppresses a tool_result, the buffer is discarded (via hook_suppressed_tool_result). One subtlety: the pending.size === 0 check means this only triggers when the suppressed result was the last pending one. For multi-tool-call assistants where a hook suppresses one result but not others, the incomplete pair persists until the next turn boundary hits new_turn_before_pair_completed. This is fine — the orphan never reaches JSONL either way — but worth a brief inline comment if you want to make the intent explicit.

  3. Legacy compat. The OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic path correctly pushes synthesized results into the buffer, clears pending, then commits. Clean fallback for environments relying on the old behavior.

  4. dropOrphanedToolPairs() for session repair. Good complement to the guard — handles already-corrupted JSONL on load. Correctly skips error/aborted stop reasons and preserves non-tool-use content blocks in mixed assistant messages.

  5. Test coverage. 5 atomic-commit cases (abort, failover, rate-limit, normal, sequential) plus the updated wait-for-idle tests covering both discard and synthetic modes. The hasOrphanToolUse helper in the test is a nice reusable invariant check.

LGTM. Clean implementation, proper edge case handling, and good test coverage.

@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from 3d5bdda to 05f4691 Compare February 19, 2026 14:57
@openclaw-barnacle openclaw-barnacle Bot removed channel: discord Channel integration: discord cli CLI command changes commands Command implementations labels Feb 19, 2026
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from 05f4691 to da36c45 Compare February 19, 2026 15:00
@openclaw-barnacle openclaw-barnacle Bot added the gateway Gateway runtime label Feb 19, 2026
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch 3 times, most recently from 1c20c8a to 80ac4ed Compare February 19, 2026 15:13
@openclaw-barnacle openclaw-barnacle Bot added the commands Command implementations label Feb 19, 2026
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from 0e78c2e to 2315445 Compare February 19, 2026 15:26
@openclaw-barnacle openclaw-barnacle Bot added cli CLI command changes and removed gateway Gateway runtime commands Command implementations labels Feb 19, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Feb 28, 2026
amabito added 4 commits March 1, 2026 15:54
…ault

Prevent orphaned tool_use blocks from being written to JSONL when a run
is interrupted (abort, failover, rate-limit). The pair buffer is held in
memory and committed atomically only when all tool_results for the turn
have been received.

Key changes:
- session-tool-result-guard.ts: introduce toolPairBuffer; assistant
  messages with tool_use are now buffered (not written) until the
  matching tool_result(s) arrive. On any interruption, the buffer is
  discarded by default (OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic
  preserves legacy behaviour). discardToolPairBuffer() logs structured
  fields: discard_reason, discarded_tool_use_count,
  discarded_tool_result_count.
- session-transcript-repair.ts: add dropOrphanedToolPairs() — strict
  removal of unmatched tool_use/tool_result pairs without synthesising
  replacements.
- session-tool-result-guard.atomic-commit.test.ts: 5 new cases
  covering abort, failover, rate-limit, normal pair, and partial
  (first pair committed / second pair aborted) scenarios.
- pi-embedded-runner.guard.waitforidle-before-flush.test.ts: update
  existing tests to reflect pair-atomic semantics (nothing written
  until pair completes) and add explicit coverage for synthetic mode.
…over coverage

Two minimal follow-up fixes after the pair-atomic commit:

1. session-tool-result-guard.ts: the hook-suppressed-result path previously
   called commitToolPairBuffer() and relied on validatePairIntegrity() to
   detect the incomplete pair and discard. This obscured the intent. Change
   to call discardToolPairBuffer("hook_suppressed_tool_result") directly,
   making it explicit that a suppressed result cannot satisfy its tool_use
   counterpart and the entire pair must be dropped.

2. attempt.ts: add a FAILOVER COVERAGE comment to the finally block that
   calls flushPendingToolResultsAfterIdle(). Documents that this is the
   single flush point for all abort/rate-limit/provider-failover exit paths:
   run.ts advances to the next auth profile only after runEmbeddedAttempt()
   returns, so incomplete pairs are always discarded before the next attempt.
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch 2 times, most recently from cf56045 to 9ba3651 Compare March 1, 2026 07:23
Existing tests expected assistant messages with tool_use to be
immediately persisted to JSONL. With pair-atomic buffering, these
messages are held until matching tool_results arrive. Updated tests
to either: set OPENCLAW_TOOL_GUARD_ABORT_MODE=synthetic with explicit
flush, or adjust expectations for discard-default semantics.
@amabito
amabito force-pushed the fix/tool-result-guard-pair-atomic branch from 9ba3651 to 7585244 Compare March 1, 2026 07:27
@amabito

amabito commented Mar 1, 2026

Copy link
Copy Markdown
Author

The remaining CI failure (checks-windows shard 2/2) is unrelated to this PR — it's a flaky test in src/cron/service.issue-regressions.test.ts ("respects abort signals while retrying main-session wake-now heartbeat runs").

  • This PR only touches src/agents/pi-embedded-runner.guard.test.ts and src/agents/session-tool-result-guard.test.ts
  • The failing test passes on Linux (both bun and node runners) and on Windows shard 1/2
  • It appears to be a timing-dependent race in the abort-signal + timeout path that fails under Windows CI scheduling pressure
  • main branch also shows intermittent CI failures in recent runs

Could a maintainer re-run the failed job? I don't have permission to trigger re-runs on this repo.

@amabito

amabito commented Mar 12, 2026

Copy link
Copy Markdown
Author

Any update on this? Happy to rebase if needed.

@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Mar 30, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #pr-thunderdome-dangerzone on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling cli CLI command changes size: L stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants