Repository navigation
fix(core): emit tool result events for user-denied HITL calls - #3104
Conversation
Signed-off-by: Yohanes <[email protected]>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
oss-maintainer
left a comment
There was a problem hiding this comment.
Summary
Emitting the missing ToolResultStartEvent / TextDeltaEvent / ToolResultEndEvent triple for user-denied HITL tool calls is the right fix, it mirrors the existing rule-denied emission in runToolBatch, and it ships with a focused regression test. Left as COMMENT because two questions are open (blank-replyId no-op, and publishing events from a synchronous helper rather than the stream).
Findings
- [Warning]
ReActAgent.java:2014— the blank-replyIdguard silently emits nothing, so the stuck-pending symptom survives for older persisted sessions. - [Warning]
ReActAgent.java:1916— events are published outside the reactive chain; please confirm subscriber ordering. - [Info]
ReActAgent.java:2022— duplicate literal for the denial reason. - [Info] test — no coverage for the blank-replyId branch or the auto-denied path.
Cross-PR Note
PR #3099 (also "emit permission-denied tool result events") overlaps heavily with this change and takes a different approach: it extracts a shared deniedToolResultEvents(...) helper, covers the auto-denied path, and mints a reply id when the correlation metadata is missing. Maintainers may want to land one of the two and fold the useful parts of the other in, rather than both.
Automated review by github-manager-bot
| ToolResultMessageBuilder.buildToolResultMsg( | ||
| deniedResult, denied, getName()); | ||
| state.contextMutable().add(deniedMsg); | ||
| if (replyId != null && !replyId.isEmpty()) { |
There was a problem hiding this comment.
Skipping emission whenever replyId is blank leaves exactly the symptom this PR fixes: for an ASKING tool call persisted before the correlation metadata existed (or after clearPendingRequestReplyId), no ToolResult* events are published, so an AG-UI client still renders a permanently pending tool call. Could this path emit under a freshly minted reply id (as PR #3099 does) or at least log at warn level so the silent no-op is diagnosable?
| replyId, | ||
| denied.getId(), | ||
| denied.getName(), | ||
| "Permission denied by user")); |
There was a problem hiding this comment.
The reason string is now duplicated in two places (ToolResultBlock.text(...) above and this delta). If they ever drift, the message written to context and the one streamed to the client disagree. Worth extracting a private static final String DENIED_BY_USER = "Permission denied by user"; and reusing it in both spots.
| } | ||
|
|
||
| applyConfirmResults(normalized); | ||
| applyConfirmResults(normalized, replyId); |
There was a problem hiding this comment.
applyConfirmResults is a synchronous state-mutation helper, so these events are pushed through publishEvent while the resume call is still validating input, i.e. outside the reactive chain that emits RequireUserConfirmEvent / RequestStopEvent. Please confirm the ordering guarantees: a subscriber that starts collecting after validateAndAcceptConfirmResults returns (or a replay from persisted state) may never observe the denied triple. Returning Flux<AgentEvent> from the helper — as runToolBatch does — would keep emission inside the stream.
| assertEquals("tc1", confirm.getConfirmResults().get(0).getToolCall().getId()); | ||
| } | ||
|
|
||
| @Test |
There was a problem hiding this comment.
Nice regression test. Two gaps worth covering: (1) resume without METADATA_CONFIRM_REQUEST_REPLY_ID (the blank-replyId branch, which today emits nothing), and (2) the rule-denied path — writeAutoDeniedResults still writes DENIED blocks without publishing events, so an auto-denied tool has the same stuck-pending symptom this PR addresses for user denials.
oss-maintainer
left a comment
There was a problem hiding this comment.
Summary
Right fix for the right symptom: a user-denied HITL tool call previously wrote a DENIED ToolResultBlock into context without ever telling the event stream, so AG-UI style consumers were left with a tool call that started and never ended. Sourcing replyId from the pending confirm request and correlating start/delta/end with the same toolCallId + ToolResultState.DENIED is exactly the contract the rule-denied path already uses, and the new test pins the ordering and the correlation. CI green on 8fa232e, CLA signed.
Caveat for the maintainers: this overlaps with PR #3099 ("fix: emit permission-denied tool result events"), which is currently in conflict with main — please land one of the two rather than both.
Findings
- [Warning]
agentscope-core/.../ReActAgent.java:2040-2055— third copy of the DENIED start/delta/end triple and of the"Permission denied by user"literal (runToolBatch()already does this for rule-denied calls); see inline comment on extracting one helper. - [Info]
ReActAgent.java:2040— the blank-replyIdguard silently skips emission and no test covers it. - [Info]
ReActAgent.java:2051— overloaded signature + unchanged javadoc leave a hook for a future caller to drop the events.
Not approving — this is review feedback only; the merge decision stays with the maintainers.
Automated review by github-manager-bot
| if (replyId != null && !replyId.isEmpty()) { | ||
| publishEvent( | ||
| new ToolResultStartEvent(replyId, denied.getId(), denied.getName())); | ||
| publishEvent( | ||
| new ToolResultTextDeltaEvent( | ||
| replyId, | ||
| denied.getId(), | ||
| denied.getName(), | ||
| "Permission denied by user")); | ||
| publishEvent( | ||
| new ToolResultEndEvent( | ||
| replyId, | ||
| denied.getId(), | ||
| denied.getName(), | ||
| ToolResultState.DENIED)); | ||
| } |
There was a problem hiding this comment.
[Performance / maintainability] This is the third copy of the "synthesise a DENIED tool-result event triple" shape — runToolBatch() already builds the identical ToolResultStartEvent / ToolResultTextDeltaEvent / ToolResultEndEvent sequence for rule-denied calls (around ReActAgent.java:3006-3021 on main), and the literal "Permission denied by user" is now duplicated with line 2033 of this file. Please extract one private helper (e.g. Flux<AgentEvent> deniedToolResultEvents(String replyId, ToolUseBlock call, String reason) or a small publishDeniedToolResult(...)), so the user-denied, rule-denied and any future deny path cannot drift apart in payload text or event ordering.
| ToolResultMessageBuilder.buildToolResultMsg( | ||
| deniedResult, denied, getName()); | ||
| state.contextMutable().add(deniedMsg); | ||
| if (replyId != null && !replyId.isEmpty()) { |
There was a problem hiding this comment.
[Correctness] The replyId != null && !replyId.isEmpty() guard is the only place in this file where event emission is skipped on a missing reply id, and the no-event branch has no test. Consequence of falling into it: the DENIED ToolResultBlock is written to context but the UI/AG-UI stream never sees a ToolResultEndEvent, which is exactly the desynchronisation this PR sets out to fix. Worth either (a) a test that pins the silent-resume behaviour, or (b) an explicit contract assertion that replyId is always non-null on this path (it comes from resolvePendingRequestReplyId() at the caller) so the guard can be dropped rather than left untested.
| "Permission denied by user")); | ||
| publishEvent( | ||
| new ToolResultEndEvent( | ||
| replyId, |
There was a problem hiding this comment.
[Compatibility] Overloading applyConfirmResults(List<ConfirmResult>) rather than changing its signature would silently leave a future caller on the single-arg version and drop the events again; the javadoc block above ("write a DENIED ToolResultBlock to context so the tool will no longer be pending on resume") also does not mention that the method now publishes tool-result events as a side effect. Since there is exactly one call site (validateAndAcceptConfirmResults), consider changing the signature outright and extending that javadoc to state the event contract.
The ubuntu build failed in ToolConfirmationCoordinatorTest .replacementTurnLeaseCannotReleaseOldTicketAndMayReuseToolUseId with "CannotStubVoidMethodWithReturnValue: 'appendSessionEvent' is a void method" while the failing line stubs managedExecutionScope. That is the signature of the shared Mockito mock being called from the coordinator's worker thread while the test thread sets up a stub: the void appendSessionEvent call is recorded and blamed instead of the stub. The test comes from upstream main (agentscope-ai#3104), this branch does not touch service-dataplane, and it passes 10/10 locally. Re-running for a clean signal.
The ubuntu build failed in ToolConfirmationCoordinatorTest .replacementTurnLeaseCannotReleaseOldTicketAndMayReuseToolUseId with "CannotStubVoidMethodWithReturnValue: 'appendSessionEvent' is a void method" while the failing line stubs managedExecutionScope. That is the signature of the shared Mockito mock being called from the coordinator's worker thread while the test thread sets up a stub: the void appendSessionEvent call is recorded and blamed instead of the stub. The test comes from upstream main (agentscope-ai#3104), this branch does not touch service-dataplane, and it passes 10/10 locally. Re-running for a clean signal.
What problem does this PR solve?
When a user denies a permission-mode HITL tool call, the resumed run writes a DENIED ToolResultBlock to context but emits no ToolResultStartEvent/ToolResultTextDeltaEvent/ToolResultEndEvent. This leaves AG-UI and other event-driven consumers unable to observe the outcome. Rule-denied calls already emit the corresponding sequence.
What changed
ToolResultState.DENIED.Verification
mvn -pl agentscope-core -DskipTests -Dspotless.check.skip=false spotless:checkmvn -pl agentscope-core -Dtest=ReActAgentHitlTest test(14 tests passed)Signed-off-by: Yohanes [email protected]