Skip to content

fix(dotnet): coordinate mixed client and server approvals - #2450

Open
javiercn wants to merge 18 commits into
ag-ui-protocol:mainfrom
javiercn:javiercn-fix-mixed-tool-approvals
Open

javiercn wants to merge 18 commits into
ag-ui-protocol:mainfrom
javiercn:javiercn-fix-mixed-tool-approvals

Conversation

@javiercn

@javiercn javiercn commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • rebase onto current main, including call-ID-aware continuation filtering from fix(dotnet): stream new tool calls on continuations #2456
  • preserve the canonical TypeScript client contract for ordinary message-only mixed continuation (C,S + result(C), no Resume)
  • use ToolApprovalRequestContent.RequiresConfirmation to distinguish genuine server approvals from collateral FICC wrapping
  • keep client-owned calls and ordinary server peers visible/renderable while executing only client-owned tools locally
  • route client approval responses through local FICC and genuine server decisions through AG-UI Resume
  • rebuild complete provider-valid call/result batches internally, correlate repeated client calls by exact call ID, and preserve original call order
  • bind every approval response and Resume decision to its original request call ID, name, and arguments
  • reject missing, crossed, stale, substituted, replayed, or invalid approval responses
  • update AGUI.Server's documented dependency on full Microsoft.Extensions.AI

Related to microsoft/agent-framework#6967 and dotnet/extensions#7549.

Canonical TS scenarios

The TS→.NET suite now covers each scenario independently:

  • client-only call: ordinary tool call, no interrupt, message-only result continuation
  • ordinary server-only call: executes and returns its result in one run
  • protected server-only call: one interrupt, explicit approve/reject Resume
  • client-local approval: application approve/reject remains message-only
  • ordinary mixed client/server: both calls rendered, client result returned without Resume, server peer executes internally
  • mixed client-local approval: approve/reject retains the ordinary message-only continuation
  • mixed protected server call: only the protected call interrupts; client result plus Resume(protected) continues ordinary peers
  • multiple protected server calls: TS requires Resume coverage for every pending interrupt

Each scenario was added as an independent commit.

Reviewer feedback addressed

  • rebased after fix(dotnet): stream new tool calls on continuations #2456 merged
  • crossed approval responses are validated against the original request and rejected
  • existing TypeScript message-only continuation contract is preserved
  • AGUI.Server dependency documentation now matches the full Microsoft.Extensions.AI reference

Validation

  • full .NET solution build: passed, 0 warnings/errors
  • signed AGUI.Client and AGUI.Server builds: passed
  • AGUI.Client.UnitTests: 133/133 on net8.0, net9.0, and net10.0
  • AGUI.Server.UnitTests: 150/150 on net8.0, net9.0, and net10.0
  • AGUI.Hosting.AspNetCore.IntegrationTests: 121/121 on net10.0
  • TS→.NET CrossLanguage.Vitest: 21 files, 133/133 tests

The cross-language suite uses the repository-managed @protobuf-ts/protoc; the conflicting external WinGet protobuf installation was removed.

@javiercn javiercn changed the title fix(dotnet): handle mixed client and server tool calls fix(dotnet): unwrap collateral server approvals Aug 18, 2026
@javiercn javiercn changed the title fix(dotnet): unwrap collateral server approvals fix(dotnet): coordinate mixed client and server approvals Aug 18, 2026
@javiercn
javiercn marked this pull request as ready for review August 21, 2026 11:17
@javiercn
javiercn requested a review from a team as a code owner August 21, 2026 11:17

@contextablemark contextablemark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested

#2456 is now merged, so the continuation-filtering dependency is resolved. Please rebase this PR onto current main and update the dependency section. I checked the combined tree; two correctness blockers remain.

1. Bind each approval response to its original request

AGUIChatClient.ValidateApprovalResponses only checks that every request ID appears in the responses. Resume construction then trusts approvalResponse.ToolCall.

With two requests, callers can cross the request IDs and tool calls. A “deny delete_file / approve get_weather” response pair is consequently serialized as “approve delete_file / deny get_weather.” Both calls exist in the original batch, so the server’s call validation accepts them.

Please map responses to requests by RequestId, verify call ID/name/arguments match, and construct the resume from the original request’s tool call. Add a regression test with crossed responses.

2. Preserve the existing cross-language continuation contract

RunAgentInputExtensions.ConfigureForMixedInvocation now rejects a mixed batch containing an AGUIToolMessage client result unless the request includes synthetic Resume entries for every peer call.

That conflicts directly with the existing TypeScript contract:

  • CrossLanguage.Vitest/tests/mixed-tool-invocation.test.ts sends the assistant batch plus the client-owned tool result and expects the C# server to execute the unresolved server tool.
  • The new LegacyResult_MixedCallBatchRequiresResume test sends that same shape and expects it to throw.

The TypeScript client cannot manufacture approval IDs for server peers that were never exposed as interrupts. Please retain the message-only continuation path—while keeping the new exact-call validation—or land a coordinated protocol and TypeScript-client change. The cross-language mixed-tool test needs to pass before merge.

Architecture follow-up

AGUI.Server now references full Microsoft.Extensions.AI, while AGENTS.md and docs/architecture.md still define it as depending only on Microsoft.Extensions.AI.Abstractions. Please either preserve that boundary or explicitly update and justify the architecture contract.

If it would be easier, I’m happy to take over this PR and handle the rebase and fixes—just let me know.

javiercn and others added 3 commits August 24, 2026 14:27
Wrap client declarations as approval-required invocable proxies, use MEAI approval causality for collateral server calls, and rebuild continuation batches with exact client results.

Co-authored-by: Copilot App <[email protected]>
Keep the existing approval flow unchanged and use RequiresConfirmation only to unwrap collateral server calls.

Co-authored-by: Copilot App <[email protected]>
Coordinate client and server approvals through complete Resume batches, preserve genuine approval decisions, and prevent partial, stale, or replayed tool execution.

Co-authored-by: Copilot App <[email protected]>
@javiercn
javiercn force-pushed the javiercn-fix-mixed-tool-approvals branch from b66d3e1 to ec558c0 Compare August 24, 2026 12:30
javiercn and others added 12 commits August 24, 2026 15:58
Prove the canonical TS client receives an ordinary tool call, no interrupt, and resumes from message history.

Co-authored-by: Copilot App <[email protected]>
Prove an ordinary server tool executes and returns its result within one successful run.

Co-authored-by: Copilot App <[email protected]>
Prove protected server tools surface one interrupt and resume through explicit approve and reject paths.

Co-authored-by: Copilot App <[email protected]>
Prove local client approval and rejection remain message-only and do not create AG-UI interrupts or Resume entries.

Co-authored-by: Copilot App <[email protected]>
Accept C+S plus the client-owned result without Resume, while rebuilding the complete internal FICC approval batch for valid provider ordering.

Co-authored-by: Copilot App <[email protected]>
Prove client-local approve and reject policies retain the ordinary C+S message-only continuation.

Co-authored-by: Copilot App <[email protected]>
Prove C+P+S emits only P as an interrupt and resumes with result(C) plus Resume(P), while ordinary S executes internally.

Co-authored-by: Copilot App <[email protected]>
Prove every protected server interrupt must be covered and complete approve/reject Resume batches continue successfully.

Co-authored-by: Copilot App <[email protected]>
Keep client approvals inside local FICC, send Resume only for genuine server interrupts, preserve informational peers and original call order, and reject crossed responses.

Co-authored-by: Copilot App <[email protected]>
Treat completed batches as continuations for replay suppression while allowing canonical message-only and interrupt-subset continuation shapes.

Co-authored-by: Copilot App <[email protected]>
Record message-only mixed continuation and genuine server-only Resume shapes after aligning the .NET client with TypeScript.

Co-authored-by: Copilot App <[email protected]>
Record the full Microsoft.Extensions.AI dependency used for exact call-ID correlation in mixed continuations.

Co-authored-by: Copilot App <[email protected]>
@javiercn

Copy link
Copy Markdown
Contributor Author

Addressed the requested changes and pushed the updated scenario-by-scenario series:

  • rebased onto current main after fix(dotnet): stream new tool calls on continuations #2456
  • bound every approval response to its original request (ID/name/arguments), with crossed-response regression coverage
  • restored and strengthened the canonical TypeScript message-only mixed continuation
  • added deterministic TS→.NET coverage for client-only, server-only, server approval, client-local approval, ordinary mixed, mixed client approval, mixed server approval, and multiple protected calls
  • aligned AGUIChatClient routing: client approvals stay in local FICC; only genuine server approvals become Resume; ordinary server peers remain informational/renderable
  • updated AGUI.Server dependency documentation

Validation is green: Client 133/TFM, Server 150/TFM, Hosting 121, CrossLanguage.Vitest 133/133. Please re-review when convenient.

@github-actions

github-actions Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Python Preview Packages

Version 0.0.0.dev1793292470 published to TestPyPI.

Warning: These packages are built from contributor code that may not yet have been vetted for correctness or security. Install at your own risk and do not use in production.

Install with uv

Add the TestPyPI index to your pyproject.toml:

[[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = true

Then install the packages you need:

# Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1793292470' --index testpypi

# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1793292470' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1793292470' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1793292470' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1793292470' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1793292470' --index testpypi

Install with pip

pip install \
  --index-url https://test.pypi.org/simple/ \
  --extra-index-url https://pypi.org/simple/ \
  ag-ui-protocol==0.0.0.dev1793292470

Use --extra-index-url https://pypi.org/simple/ so pip can resolve
transitive dependencies (pydantic, fastapi, etc.) from real PyPI.


Commit: 2f45d24

@pkg-pr-new

pkg-pr-new Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@ag-ui/a2a-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a-middleware@2450

@ag-ui/a2ui-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-middleware@2450

@ag-ui/event-throttle-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/event-throttle-middleware@2450

@ag-ui/mcp-apps-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-apps-middleware@2450

@ag-ui/mcp-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-middleware@2450

@ag-ui/a2a

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a@2450

@ag-ui/adk-js

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk-js@2450

@ag-ui/adk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk@2450

@ag-ui/ag2

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/ag2@2450

@ag-ui/agno

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/agno@2450

@ag-ui/antigravity

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/antigravity@2450

@ag-ui/aws-strands

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/aws-strands@2450

@ag-ui/claude-agent-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-agent-sdk@2450

@ag-ui/claude-managed-agents

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-managed-agents@2450

@ag-ui/crewai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/crewai@2450

@ag-ui/langchain

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langchain@2450

@ag-ui/langgraph

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langgraph@2450

@ag-ui/llamaindex

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/llamaindex@2450

@ag-ui/mastra

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mastra@2450

@ag-ui/pydantic-ai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/pydantic-ai@2450

@ag-ui/vercel-ai-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/vercel-ai-sdk@2450

@ag-ui/watsonx

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/watsonx@2450

@ag-ui/a2ui-toolkit

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-toolkit@2450

create-ag-ui-app

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/create-ag-ui-app@2450

@ag-ui/client

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/client@2450

@ag-ui/core

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/core@2450

@ag-ui/encoder

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/encoder@2450

@ag-ui/proto

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/proto@2450

commit: e4f0beb

@contextablemark contextablemark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested

Thanks for the thorough follow-up. I re-reviewed the latest head (d9dd9ac) and confirmed that the issues from my first review are addressed: the branch is rebased after #2456, approval responses are bound to their original calls, the TypeScript message-only continuation works again, replay suppression is restored, and the full Microsoft.Extensions.AI dependency is now documented.

I also reran the advertised suites locally. The solution builds cleanly, Client passes 133/133, Server 150/150, Hosting 121/121, C#→TypeScript 20/20, and TypeScript→C# 133/133. However, the cross-language TypeScript typecheck currently fails as described below.

Two approval-flow correctness blockers remain, and the new cross-language test file does not typecheck.

1. Reject a partial Resume before rebuilding the approval batch

IsResumeForLatestCallBatch currently checks only that every resumed call belongs to the latest batch. It does not require every unresolved call in that batch to be covered by either a persisted result or a Resume entry.

ProcessContinuation then creates a RequiresConfirmation = false request and an approved response for each omitted call. Consequently, when a batch contains two protected server calls, a caller can send one fully formed Resume and omit the other; the omitted protected call is silently approved and executed.

I reproduced this with two ApprovalRequiredAIFunction server tools. Resuming only the first call resulted in the omitted second tool executing once; the expected count was zero.

The new “requires Resume for every pending interrupt” TypeScript test does not cover this server-side path: its partial payload omits toolCall, so HttpAgent rejects it from local pendingInterrupts before a request reaches the .NET server. Please add a server-level or raw-HTTP regression using two protected calls and one otherwise-valid, fully shaped Resume, and reject the request before either pending call is executed.

2. Do not require responses for approvals already resolved in history

The client correctly collects fresh approval responses only from the last message, but ValidateApprovalResponses collects approval requests from the entire message history and requires every one of them to appear in that fresh-response set.

This breaks the second approval round in the SDK's full-history model. I reproduced it with an already-resolved old request/response followed by a new request/response; the new turn throws:

Approval responses are missing for request(s): approval-old.

Please validate only the current unresolved approval batch, or account for historical responses when determining which requests remain unanswered, while retaining the new exact request/call binding checks.

3. Make the new cross-language tests pass typecheck

The current CI typecheck job is red, and I reproduced it locally with:

nx run @ag-ui/cross-language-tests:typecheck

All 13 diagnostics are TS2322 errors in tool-approval-scenarios.test.ts. Each expression-bodied callback has the form:

onEvent: ({ event }) => turn1.push(event)

Array.push returns a number, so these callbacks do not satisfy the subscriber contract's void | AgentStateMutation return type. Using a block body or explicitly discarding the result fixes the target; after changing only these callbacks to ({ event }) => void turn1.push(event), the exact Nx target passes.

Additional cleanup

  • RawRepresentationFactory is invoked once for the Resume preflight and again by the inner handler (first call, second call). A counting regression sees two invocations, and a factory that produces fresh input can make preflight routing inspect a different RunAgentInput from the one sent. Please evaluate it once and carry that value through.
  • dotnet format --verify-no-changes rejects AGUIChatClient.cs, primarily the block at lines 97–130, and reports three unused imports.

My offer to take over the remaining fixes still stands if that would be easier.

@contextablemark contextablemark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The partial-Resume fix, historical-response handling, and TypeScript typecheck check out in the targeted runs. One approval gap remains; I’ve left it inline. This is an existing path left open, rather than a regression introduced by the latest merge.

#pragma warning restore MEAI001
};
approvalRequests.Add(request);
approvalResponses.Add(request.CreateResponse(approved: true));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The partial-Resume fix works, but omitting Resume entirely still skips approval.

I reproduced this on 1d9762ce0e03eca26377ec6461f3765178b025be with one completed client result and two server calls wrapped in ApprovalRequiredAIFunction:

  • One valid Resume: rejected; neither protected tool executes.
  • No Resume: accepted; both protected tools execute once.

With no approval Resume, the request takes the older message-only continuation path, and this line supplies an approved response for each protected call. That path was already present at the previously reviewed head, so this is a remaining gap rather than a regression from this merge.

Please keep message-only continuation working for ordinary server peers, while requiring a decision before protected tools execute. Could you add the same server-level regression with the client result present and Resume omitted, asserting zero protected executions?

I haven’t included a one-click patch because the actual server tools are registered after this adapter runs; a safe fix needs to distinguish their approval requirements without breaking ordinary mixed continuation.

@contextablemark contextablemark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the direct protected-tool case is fixed now. I confirmed protected peers no longer run without Resume, whether they're passed in, added afterward, or only in AdditionalTools. There's one remaining gap, noted inline. The other inline note is non-blocking.

// Wire history cannot establish whether a server tool requires approval.
// Only the application's trusted registrations can authorize the legacy path.
var tool = chatOptions.Tools?.SingleOrDefault(tool => tool.Name == call.Name);
if (tool is not AIFunction || tool is ApprovalRequiredAIFunction)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This catches ApprovalRequiredAIFunction itself, but not one that's been wrapped in another DelegatingAIFunction (a logging or telemetry wrapper, say). The wrapped tool passes this check as ordinary, gets a synthetic approval, and runs with no Resume. FICC still treats the same wrapper as needing approval on a normal call, so the two disagree.

I reproduced it on 54bb333 with new Wrap(new ApprovalRequiredAIFunction(f)), where Wrap : DelegatingAIFunction. With a client result and zero Resume entries, the protected function ran once.

Checking it the way FICC does should close it:

if (tool is not AIFunction || tool.GetService<ApprovalRequiredAIFunction>() is not null)

Could you also add a wrapped case to LegacyResult_ProtectedServerPeersCannotExecuteWithoutResume?


// Wire history cannot establish whether a server tool requires approval.
// Only the application's trusted registrations can authorize the legacy path.
var tool = chatOptions.Tools?.SingleOrDefault(tool => tool.Name == call.Name);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: if two registered tools share a name, this throws LINQ's "Sequence contains more than one matching element". It fails safe, so it's fine as is, but a message naming the ambiguous tool would be easier to debug.

@Laptopmini

Copy link
Copy Markdown
Contributor

This also fixes #3271 (a client tool that already has a result loses its schema and description on the next continuation, and its stand-in answers new calls with the old result). In case they're useful, here are regression tests for that issue, written against this PR's API:

Test main (366b4f8) this PR (e4f0beb)
Continuation_ClientToolWithEarlierResult_KeepsItsParameterSchema fails: description "", no colour property passes
Continuation_ClientToolWithEarlierResult_DoesNotAnswerNewCallsWithTheEarlierResult fails: returns "Applied red" for a request for blue passes
Continuation_NewCallToResolvedClientTool_ReachesTheClientWithItsArguments (FICC + AsAGUIEventStreamAsync) fails: the model is offered no colour parameter passes
MixedContinuation_ClientToolCalledAgainAfterServerTool_ReachesTheClient (guard) passes passes

Run with .NET SDK 10.0.100, net10.0: dotnet test tests/AGUI.Server.UnitTests -c Release -p:SignAssembly=false --filter "FullyQualifiedName~ContinuationClientToolDeclarationTest".

The only change needed for this PR is in RunTurn: the guard test registers its server tool through the new serverTools overload. Without that, ToChatRequestContext throws "requires an explicit Resume decision or a trusted ordinary server tool registration", as intended. Feel free to take the file as is, adapt it, or ignore it. If you add it, please add Fixes #3271 to the description.

sdks/dotnet/tests/AGUI.Server.UnitTests/ContinuationClientToolDeclarationTest.cs
using System.Runtime.CompilerServices;
using System.Text.Json;
using AGUI.Abstractions;
using Microsoft.Extensions.AI;
using Xunit;

namespace AGUI.Server.UnitTests;

/// <summary>
/// Regression coverage for ag-ui#3271: on a continuation, a client tool that already produced a
/// result must stay declared exactly as the client declared it. Resolving one call must not
/// change the schema or the execution owner of later calls to the same tool. Each new call has
/// to reach the client with its own arguments, and must never be answered on the server with
/// the earlier call's result.
/// </summary>
public sealed class ContinuationClientToolDeclarationTest
{
    private static readonly JsonSerializerOptions SerializerOptions = AIJsonUtilities.DefaultOptions;

    private const string ThreadId = "thread-1";
    private const string RunId = "run-1";

    private static readonly JsonElement ColourSchema = JsonDocument.Parse(
        """{"type":"object","properties":{"colour":{"type":"string"}},"required":["colour"]}""")
        .RootElement.Clone();

    [Fact]
    public void Continuation_ClientToolWithEarlierResult_KeepsItsParameterSchema()
    {
        var context = BuildContinuationInput().ToChatRequestContext(SerializerOptions);

        var declaration = Assert.IsAssignableFrom<AIFunctionDeclaration>(Assert.Single(context.ChatOptions.Tools!));
        Assert.Equal("set_colour", declaration.Name);
        Assert.Equal("Change the UI colour.", declaration.Description);
        Assert.True(declaration.JsonSchema.GetProperty("properties").TryGetProperty("colour", out _));
        Assert.Equal("colour", declaration.JsonSchema.GetProperty("required")[0].GetString());
    }

    /// <summary>
    /// Whatever the adapter installs for the client tool, invoking it for a call other than the
    /// resolved one must not produce the earlier call's result: a new call belongs to the client.
    /// </summary>
    [Fact]
    public async Task Continuation_ClientToolWithEarlierResult_DoesNotAnswerNewCallsWithTheEarlierResult()
    {
        var tool = Assert.Single(BuildContinuationInput().ToChatRequestContext(SerializerOptions).ChatOptions.Tools!);
        if (tool is not AIFunction function)
        {
            return; // Declaration only: nothing on the server can answer the call.
        }

        object? result;
        try
        {
            result = await function.InvokeAsync(new AIFunctionArguments(
                new Dictionary<string, object?> { ["colour"] = "blue" }));
        }
        catch (InvalidOperationException)
        {
            return; // Refusing a call it has no result for is also correct.
        }

        Assert.NotEqual("Applied red", result?.ToString());
    }

    /// <summary>
    /// The reported scenario, driven through FunctionInvokingChatClient: after "Applied red", the
    /// model asks for blue. The model must see the colour parameter, and the new call must reach
    /// the client as a plain TOOL_CALL carrying its own arguments, with no server-side result.
    /// </summary>
    [Fact]
    public async Task Continuation_NewCallToResolvedClientTool_ReachesTheClientWithItsArguments()
    {
        var llm = new ScriptedChatClient(
            new FunctionCallContent("call-new", "set_colour", new Dictionary<string, object?> { ["colour"] = "blue" }));

        var events = await RunTurn(llm, serverTools: [], BuildContinuationInput());

        var advertised = Assert.IsAssignableFrom<AIFunctionDeclaration>(Assert.Single(llm.ToolsSeen[0]));
        Assert.True(advertised.JsonSchema.GetProperty("properties").TryGetProperty("colour", out _));

        AssertSurfacedToClient(events, "call-new", "blue");
        Assert.IsType<RunFinishedSuccessOutcome>(events.OfType<RunFinishedEvent>().Single().Outcome);
        Assert.Equal(1, llm.Calls);
    }

    /// <summary>
    /// Several UI actions in one request: the continuation resolves the earlier client call and
    /// runs a pending server tool, then the model calls the client tool again in the same run.
    /// That second call must still go to the client, not be answered with "Applied red".
    /// </summary>
    [Fact]
    public async Task MixedContinuation_ClientToolCalledAgainAfterServerTool_ReachesTheClient()
    {
        var serverToolInvoked = false;
        var serverTools = new List<AITool>
        {
            AIFunctionFactory.Create(() => { serverToolInvoked = true; return "table shown"; }, "show_table"),
        };

        var llm = new ScriptedChatClient(
            new FunctionCallContent("call-dock", "set_colour", new Dictionary<string, object?> { ["colour"] = "blue" }));

        var input = BuildContinuationInput();
        var assistant = (AGUIAssistantMessage)input.Messages[1];
        assistant.ToolCalls!.Add(new AGUIToolCall
        {
            Id = "call-server",
            Function = new AGUIToolCallFunction { Name = "show_table", Arguments = "{}" }
        });
        input.Messages.RemoveAt(input.Messages.Count - 1);

        var events = await RunTurn(llm, serverTools, input);

        Assert.True(serverToolInvoked, "the pending server tool should run on the continuation");
        Assert.Contains(events.OfType<ToolCallResultEvent>(), e => e.ToolCallId == "call-server");
        AssertSurfacedToClient(events, "call-dock", "blue");
        Assert.IsType<RunFinishedSuccessOutcome>(events.OfType<RunFinishedEvent>().Single().Outcome);
    }

    private static void AssertSurfacedToClient(List<BaseEvent> events, string callId, string colour)
    {
        var start = Assert.Single(events.OfType<ToolCallStartEvent>(), e => e.ToolCallId == callId);
        Assert.Equal("set_colour", start.ToolCallName);

        var args = Assert.Single(events.OfType<ToolCallArgsEvent>(), e => e.ToolCallId == callId);
        using var parsed = JsonDocument.Parse(args.Delta);
        Assert.Equal(colour, parsed.RootElement.GetProperty("colour").GetString());

        Assert.DoesNotContain(events.OfType<ToolCallResultEvent>(), e => e.ToolCallId == callId);
    }

    private static RunAgentInput BuildFirstTurnInput() =>
        new()
        {
            ThreadId = ThreadId,
            RunId = RunId,
            Tools = [new AGUITool { Name = "set_colour", Description = "Change the UI colour.", Parameters = ColourSchema }],
            Messages = [new AGUIUserMessage { Content = "Make it red." }]
        };

    /// <summary>
    /// The continuation from the issue: an earlier set_colour call answered by the client with
    /// "Applied red", then a new user request for blue.
    /// </summary>
    private static RunAgentInput BuildContinuationInput()
    {
        var input = BuildFirstTurnInput();
        input.Messages =
        [
            .. input.Messages,
            new AGUIAssistantMessage
            {
                ToolCalls =
                [
                    new AGUIToolCall
                    {
                        Id = "call-old",
                        Function = new AGUIToolCallFunction { Name = "set_colour", Arguments = """{"colour":"red"}""" }
                    }
                ]
            },
            new AGUIToolMessage { ToolCallId = "call-old", Content = "Applied red" },
            new AGUIUserMessage { Content = "Now change it to blue." }
        ];
        return input;
    }

    private static async Task<List<BaseEvent>> RunTurn(
        ScriptedChatClient llm, IList<AITool> serverTools, RunAgentInput input)
    {
        // Server tools are registered through the adapter so a continuation may resume them.
        var context = input.ToChatRequestContext(SerializerOptions, null, serverTools);

        var ficc = new ChatClientBuilder(llm).UseFunctionInvocation().Build();
        var events = new List<BaseEvent>();
        await foreach (var evt in ficc.GetStreamingResponseAsync(context.Messages, context.ChatOptions)
            .AsAGUIEventStreamAsync(context).ConfigureAwait(false))
        {
            events.Add(evt);
        }

        return events;
    }

    /// <summary>
    /// A fake model: the first call proposes the given tool calls, every later call answers with
    /// text. Records the client tools it was offered on each call.
    /// </summary>
    private sealed class ScriptedChatClient(params AIContent[] firstCallContents) : IChatClient
    {
        public int Calls { get; private set; }

        public List<List<AITool>> ToolsSeen { get; } = [];

        public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
            IEnumerable<ChatMessage> messages,
            ChatOptions? options = null,
            [EnumeratorCancellation] CancellationToken cancellationToken = default)
        {
            await Task.Yield();
            ToolsSeen.Add(options?.Tools?.Where(t => t.Name == "set_colour").ToList() ?? []);
            if (Calls++ == 0)
            {
                yield return new ChatResponseUpdate
                {
                    Role = ChatRole.Assistant,
                    Contents = [.. firstCallContents],
                    FinishReason = ChatFinishReason.ToolCalls,
                };
                yield break;
            }

            yield return new ChatResponseUpdate { Role = ChatRole.Assistant, Contents = [new TextContent("done")] };
        }

        public Task<ChatResponse> GetResponseAsync(
            IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default) =>
            throw new NotSupportedException();

        public object? GetService(Type serviceType, object? serviceKey = null) => null;

        public void Dispose()
        {
        }
    }
}

@contextablemark

Copy link
Copy Markdown
Contributor

@javiercn thanks, e4f0beb covers both of my earlier comments. Wrapped protected tools are caught now, and the duplicate-name error is clear. I ran the Server unit tests on it: 222 passed.

Two things before this can go in:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants