.NET sample whatif it's NOT in Azure?

As dev I'm using AI a bit everywhere, not only Azure. Is it supose to work just the same let's say by switch the AzureOpenAIClient to OpenAIClient and specifying my Endpoint?

ex:

// Create the agent using Azure OpenAI.
// AIAgent agent = new AzureOpenAIClient(
//     new Uri(endpoint),
//     new AzureCliCredential())
//         .GetChatClient(deploymentName)
//         .CreateAIAgent(agentOptions);

AIAgent agent = new OpenAIClient(new ApiKeyCredential(API_KEY), new OpenAIClientOptions { Endpoint = new Uri(endpoint) })
                    .GetChatClient(deploymentName)
                    .CreateAIAgent(agentOptions);

var response = await agent.RunAsync("Please provide information about John Smith, who is a 35-year-old software engineer.");

There's no need for it to be Azure at all. AIAgent is just an abstract base class, which could be backed by anything. And ChatClientAgent, as one of the most prominent derivations, is just backed by an IChatClient, which has implementations for OpenAI, AWS, Gemini, Anthropic, ONNX Runtime GenAI, Hugging Face, and so on.

https://github.com/microsoft/agent-framework/tree/main/dotnet/samples/GettingStarted/AgentProviders has examples with various providers.

And https://github.com/microsoft/agent-framework/blob/main/dotnet/samples/GettingStarted/Workflows/_Foundational/05_MultiModelService/Program.cs provides an example of multiple providers working together in a workflow.

In your example, yes, you can use OpenAIClient, with whatever Endpoint you want.

I'm having mixed results using a IChatClient implementation from the Anthropic.SDK nuget package. It's mostly works, but when a TurnToken is sent to the agent based on it, it's causing a call to GetStreamingResponseAsync with an empty list of messages which ends up causing an error.

When I switch to using a OpenAI based IChatClient backed by GitHub Models that call to GetStreamingResponseAsync doesn't happen.

I'm going to try to do some more debugging on Monday, but curious if anyone else is seeing something similar.

I'll plan to either post my findings or at least sample code to reproduce the issue here early next week.

I'll plan to either post my findings or at least sample code to reproduce the issue here early next week.

That'd be great. There may be some small issues to fix in workflows, in Anthropic.SDK, or both.

I found may issue :')
I had a Azure.AI.OpenAI in the mix I think there was namespace confusion or something. It works now.

Here is some sample code that shows the problem I'm seeing. When trying to create a sequential workflow using WorkflowBuilder and the agent based on Anthropic.SDK a System.Reflection.TargetInvocationException: 'Error invoking handler for Microsoft.Agents.AI.Workflows.TurnToken' is thrown. I can see that when using AgentWorkflowBuilder there is a 3rd executor in the workflow, and my code using WorkflowBuilder only results in a two executer workflow, but that it still works when the underlying IChatClient instances are OpenAIClient I don't think that's the issue.

using Anthropic.SDK;
using Anthropic.SDK.Constants;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using OpenAI;
using System.ClientModel;

var host = Host
    .CreateApplicationBuilder(new HostApplicationBuilderSettings{EnvironmentName = "Development"})
    .Build();
var config = host.Services.GetRequiredService<IConfiguration>();


var githubModelsKey = config["GitHubModels:ApiKey"];
var openAIClient = new OpenAIClient(new ApiKeyCredential(githubModelsKey), new OpenAIClientOptions
    {
        Endpoint = new Uri("https://models.inference.ai.azure.com")
    })
    .GetChatClient("gpt-4o-mini")
    .AsIChatClient();

var anthropicKey = config["Anthropic:ApiKey"];
var anthropicClient = new AnthropicClient(new APIAuthentication(anthropicKey))
    .Messages
    .AsBuilder()
    .ConfigureOptions(options =>
    {
        options.ModelId = AnthropicModels.Claude35Haiku;
        options.MaxOutputTokens = 512;
    })
    .Build();

// Define agents using both clients
var openAIJoker = new ChatClientAgent(openAIClient, instructions:"You like to tell jokes.", name: "Joker");
var openAIAnnoyer = new ChatClientAgent(openAIClient, instructions: "You like to explain jokes.", name: "Annoyer");

var anthropicJoker = new ChatClientAgent(anthropicClient, instructions: "You like to tell jokes.", name: "Joker");
var anthropicAnnoyer = new ChatClientAgent(anthropicClient, instructions: "You like to explain jokes.", name: "Annoyer");


// Define workflows using AgentWorkflowBuilder
var openAIAgentWorkflow = AgentWorkflowBuilder.BuildSequential(openAIJoker, openAIAnnoyer);
var anthropicAgentWorkflow = AgentWorkflowBuilder.BuildSequential(anthropicJoker, anthropicAnnoyer);

// Define workflows using WorkflowBuilder
var openAIWorkflow = new WorkflowBuilder(openAIJoker)
    .AddEdge(openAIJoker, openAIAnnoyer)
    .Build();

var anthropicWorkflow = new WorkflowBuilder(anthropicJoker)
    .AddEdge(anthropicJoker, anthropicAnnoyer)
    .Build();

var prompt = "Tell me a joke about cats.";

// The first three work
Console.WriteLine("OpenAI using Workflow from AgentWorkflowBuilder:");
var openAIAgentResponse = await (await openAIAgentWorkflow.AsAgentAsync()).RunAsync(prompt);
Console.WriteLine(openAIAgentResponse.Text);
Console.WriteLine();

Console.WriteLine("Anthropic using Workflow from AgentWorkflowBuilder:");
var anthropicAgentResponse = await (await anthropicAgentWorkflow.AsAgentAsync()).RunAsync(prompt);
Console.WriteLine(anthropicAgentResponse.Text);
Console.WriteLine();

Console.WriteLine("OpenAI using Workflow from WorkflowBuilder:");
var openAIWorkflowResponse = await (await openAIWorkflow.AsAgentAsync()).RunAsync(prompt);
Console.WriteLine(openAIWorkflowResponse.Text);
Console.WriteLine();

// But this one throws an exception:
Console.WriteLine("Anthropic using Workflow from WorkflowBuilder:");
var anthropicWorkflowResponse = await (await anthropicWorkflow.AsAgentAsync()).RunAsync(prompt);
Console.WriteLine(anthropicWorkflowResponse.Text);
Console.WriteLine();

I believe I've found the beginning of tracking down where the problem I'm having is coming from. In AIAgentHostExector.TakeTurnAsync there is a check to ignore any updates that don't have a MessageId set. What's I'm seeing is that when the underlying provider is from Anthropic.SDK the MessageId is always null even when the update is relevant and has content. Because all the response updates are ignored, it ends up sending a TurnToken message with no preceding ChatMessage message.

agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIAgentHostExecutor.cs

Lines 52 to 66 in 6285419

    
           protected override async ValueTask TakeTurnAsync(List<ChatMessage> messages, IWorkflowContext context, bool? emitEvents, CancellationToken cancellationToken = default) 
        
           { 
        
               emitEvents ??= this._emitEvents; 
        
               IAsyncEnumerable<AgentRunResponseUpdate> agentStream = this._agent.RunStreamingAsync(messages, this.EnsureThread(context), cancellationToken: cancellationToken); 
        
               List<AIContent> updates = []; 
        
               ChatMessage? currentStreamingMessage = null; 
        
               await foreach (AgentRunResponseUpdate update in agentStream.ConfigureAwait(false)) 
        
               { 
        
                   if (string.IsNullOrEmpty(update.MessageId)) 
        
                   { 
        
                       // Ignore updates that don't have a message ID. 
        
                       continue; 
        
                   }

I'm not sure if that if statement is making a bad assumption about how providers use MessageId, the Anthropic.SDK code isn't, but should be passing along a MessageId, if Anthropic's API just doesn't have this concept of MessageId and so the Anthropic.SDK code should be generating one to be compliant with IChatClient.

Michael Holczer (@mikeholczer)

Thanks, Michael Holczer (@mikeholczer).

AgentWorkflowBuilder.BuildSequential doesn't use AIAgentHostExecutor, because AIAgentHostExecutor currently has a few issues (one of which you noted):

It's ignoring messages that don't have a MessageId set. westey (@westey-m), you were following up on this one I believe?
It's only forwarding on the output messages from the agent but not the input messages, which means the next agent in the conversation doesn't see the full history.
An agent is seeing all output from the previous agent as "Assistant" messages, which can confuse some providers, as the content may not be in a format they understand.

Instead, it's using this one:

agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows/AgentWorkflowBuilder.cs

Lines 147 to 193 in 6285419

    
           private sealed class AgentRunStreamingExecutor(AIAgent agent, bool includeInputInOutput) : Executor(GetDescriptiveIdFromAgent(agent)), IResettableExecutor 
        
           { 
        
               private readonly List<ChatMessage> _pendingMessages = []; 
        
               protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) => 
        
                   routeBuilder 
        
                       .AddHandler<string>((message, context) => this._pendingMessages.Add(new(ChatRole.User, message))) 
        
                       .AddHandler<ChatMessage>((message, context) => this._pendingMessages.Add(message)) 
        
                       .AddHandler<IEnumerable<ChatMessage>>((messages, _) => this._pendingMessages.AddRange(messages)) 
        
                       .AddHandler<ChatMessage[]>((messages, _) => this._pendingMessages.AddRange(messages)) // TODO: Remove once https://github.com/microsoft/agent-framework/issues/782 is addressed 
        
                       .AddHandler<List<ChatMessage>>((messages, _) => this._pendingMessages.AddRange(messages))  // TODO: Remove once https://github.com/microsoft/agent-framework/issues/782 is addressed 
        
                       .AddHandler<TurnToken>(async (token, context) => 
        
                       { 
        
                           List<ChatMessage> messages = [.. this._pendingMessages]; 
        
                           this._pendingMessages.Clear(); 
        
                           List<ChatMessage>? roleChanged = ChangeAssistantToUserForOtherParticipants(agent.DisplayName, messages); 
        
                           List<AgentRunResponseUpdate> updates = []; 
        
                           await foreach (var update in agent.RunStreamingAsync(messages).ConfigureAwait(false)) 
        
                           { 
        
                               updates.Add(update); 
        
                               if (token.EmitEvents is true) 
        
                               { 
        
                                   await context.AddEventAsync(new AgentRunUpdateEvent(this.Id, update)).ConfigureAwait(false); 
        
                               } 
        
                           } 
        
                           ResetUserToAssistantForChangedRoles(roleChanged); 
        
                           if (!includeInputInOutput) 
        
                           { 
        
                               messages.Clear(); 
        
                           } 
        
                           messages.AddRange(updates.ToAgentRunResponse().Messages); 
        
                           await context.SendMessageAsync(messages).ConfigureAwait(false); 
        
                           await context.SendMessageAsync(token).ConfigureAwait(false); 
        
                       }); 
        
               public ValueTask ResetAsync() 
        
               { 
        
                   this._pendingMessages.Clear(); 
        
                   return default; 
        
               } 
        
           }

that was put in place temporarily to address those issues. I'd like to see us address those issues in AIAgentHostExecutor and then this variation can be removed. cc: Jacob Alber (@lokitoth)

I will find time today to try out your repro and see what's going on.

Stephen Toub (@stephentoub)

Thanks Stephen Toub (@stephentoub). I just checked my logs for the streaming responses when using Anthropic's API vs GitHub Models and it looks like Anthropic's response only includes an id property for the message_start event, but not the content_block_delta events, but I don't know if that makes it correct for the updates to not have their MessageId property set, or if Anthropic.SDK should be using the id from the message_start one each update object.

Tao Chen (@TaoChenOSU)

Tao Chen (@TaoChenOSU), it looks like there are some issues with filtering out the updates without message ids for some providers.
I remember we were saying the right fix is to switch to combining updates into messages using the IEnumerable<AgentRunResponseUpdate>.ToAgentRunResponse extension method instead of the custom code currently in Workflows. Is that right?

The issue we were facing was that there could be potentially multiple messages from the agent per run invocation. Without relying on the message ID, we couldn't correctly identify the updates that belong to the same message and build the fully formed message.

I saw that IEnumerable<AgentRunResponseUpdate>.ToAgentRunResponse actually handled that. Could you confirm?

The issue we were facing was that there could be potentially multiple messages from the agent per run invocation. Without relying on the message ID, we couldn't correctly identify the updates that belong to the same message and build the fully formed message.

I saw that IEnumerable<AgentRunResponseUpdate>.ToAgentRunResponse actually handled that. Could you confirm?

ToAgentRunResponse looks for changes in message ID or role to help identify message boundaries (we should also consider updating it to factor in author name).

Michael Holczer (@mikeholczer)

Hi Michael Holczer (@mikeholczer),

The issue should have been resolved by PR #1439. Could you please try running your code again using the latest release when it comes out and let us know if it resolves the issue?

Tao Chen (@TaoChenOSU)

Tao Chen (@TaoChenOSU) Sure. Any idea when that's likely to be?

Hi @mikeholczer,

The issue should have been resolved by PR #1439. Could you please try running your code again using the latest release when it comes out and let us know if it resolves the issue?

I updated the code to the 1.0.0-preview.25.10.28.1 version of Microsoft.Agents.AI packages and Anthropic.SDK 5.8.0 and all the cases work now. Thanks.

Roger Barreto (@rogerbarreto)

Hi Roger Barreto (@rogerbarreto)

Thanks for the consideration of the community-driven NuGet package.

Cheers

Navigation Menu

Repository navigation

.NET sample whatif it's NOT in Azure? #1137

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	protected override async ValueTask TakeTurnAsync(List<ChatMessage> messages, IWorkflowContext context, bool? emitEvents, CancellationToken cancellationToken = default)
	{
	emitEvents ??= this._emitEvents;
	IAsyncEnumerable<AgentRunResponseUpdate> agentStream = this._agent.RunStreamingAsync(messages, this.EnsureThread(context), cancellationToken: cancellationToken);

	List<AIContent> updates = [];
	ChatMessage? currentStreamingMessage = null;

	await foreach (AgentRunResponseUpdate update in agentStream.ConfigureAwait(false))
	{
	if (string.IsNullOrEmpty(update.MessageId))
	{
	// Ignore updates that don't have a message ID.
	continue;
	}

	private sealed class AgentRunStreamingExecutor(AIAgent agent, bool includeInputInOutput) : Executor(GetDescriptiveIdFromAgent(agent)), IResettableExecutor
	{
	private readonly List<ChatMessage> _pendingMessages = [];

	protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) =>
	routeBuilder
	.AddHandler<string>((message, context) => this._pendingMessages.Add(new(ChatRole.User, message)))
	.AddHandler<ChatMessage>((message, context) => this._pendingMessages.Add(message))
	.AddHandler<IEnumerable<ChatMessage>>((messages, _) => this._pendingMessages.AddRange(messages))
	.AddHandler<ChatMessage[]>((messages, _) => this._pendingMessages.AddRange(messages)) // TODO: Remove once https://github.com/microsoft/agent-framework/issues/782 is addressed
	.AddHandler<List<ChatMessage>>((messages, _) => this._pendingMessages.AddRange(messages)) // TODO: Remove once https://github.com/microsoft/agent-framework/issues/782 is addressed
	.AddHandler<TurnToken>(async (token, context) =>
	{
	List<ChatMessage> messages = [.. this._pendingMessages];
	this._pendingMessages.Clear();

	List<ChatMessage>? roleChanged = ChangeAssistantToUserForOtherParticipants(agent.DisplayName, messages);

	List<AgentRunResponseUpdate> updates = [];
	await foreach (var update in agent.RunStreamingAsync(messages).ConfigureAwait(false))
	{
	updates.Add(update);
	if (token.EmitEvents is true)
	{
	await context.AddEventAsync(new AgentRunUpdateEvent(this.Id, update)).ConfigureAwait(false);
	}
	}

	ResetUserToAssistantForChangedRoles(roleChanged);

	if (!includeInputInOutput)
	{
	messages.Clear();
	}

	messages.AddRange(updates.ToAgentRunResponse().Messages);

	await context.SendMessageAsync(messages).ConfigureAwait(false);
	await context.SendMessageAsync(token).ConfigureAwait(false);
	});

	public ValueTask ResetAsync()
	{
	this._pendingMessages.Clear();
	return default;
	}
	}

.NET sample whatif it's NOT in Azure? #1137

Description

Activity

stephentoub commented on Oct 3, 2025

mikeholczer commented on Oct 3, 2025

stephentoub commented on Oct 4, 2025

fboucher commented on Oct 4, 2025

mikeholczer commented on Oct 4, 2025

mikeholczer commented on Oct 6, 2025

stephentoub commented on Oct 6, 2025

mikeholczer commented on Oct 6, 2025

westey-m commented on Oct 6, 2025

TaoChenOSU commented on Oct 7, 2025

stephentoub commented on Oct 7, 2025

TaoChenOSU commented on Oct 21, 2025

mikeholczer commented on Oct 22, 2025

mikeholczer commented on Oct 31, 2025

jochenkirstaetter commented on Dec 5, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions