.Net Can I specify the Openai service instance used by the Agent?

In the kernel I will register multiple AzureOpenAI, they represent different models that have token limitations,

IKernelBuilder builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(...MODEL 1..  ,serviceid:"ser-a");
builder.AddAzureOpenAIChatCompletion(...MODEL 2..,serviceid:"ser-b");

I have two agents and I want to specify the serviceid used by each agent.

ChatCompletionAgent agent1 = new()
{
              ExecutionSettings = new OpenAIPromptExecutionSettings()
              {
                  ExtensionData = new Dictionary<String, Object>()
      {
                      { "ser-b" , new PromptExecutionSettings() }  
      },
                  Temperature = 0.7,
                  ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
              },
              Instructions = "...........",
              Name = "ag1",
          };

Package

"Microsoft.Extensions.Logging.Console" Version="8.0.0"
"Microsoft.SemanticKernel" Version="1.13.0"
"Microsoft.SemanticKernel.Agents.Abstractions" Version="1.13.0-alpha"
"Microsoft.SemanticKernel.Agents.Core" Version="1.13.0-alpha"
"Microsoft.SemanticKernel.Agents.OpenAI" Version="1.13.0-alpha"
"Microsoft.SemanticKernel.Connectors.OpenAI" Version="1.13.0"
"Microsoft.SemanticKernel.Plugins.Core" Version="1.13.0-alpha"
"Microsoft.SemanticKernel.PromptTemplates.Handlebars" Version="1.13.0"

strikene

strikene, thank you for this insightful question. We have a variety of cases that warrant the support your are proposing that have high priority for further exploration / support.

As you've identified, the current implementation does not currently utilize service selection by id/key. This results in default service-selection logic in Kernel: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/Kernel.cs#L229

Taking a step back, your insight seems to intersect the perennial question on whether to use Kernel as a singleton. There are certainly compelling scenarios to consuming a shared kernel with multiple keyed-services. Ultimately, Kernel can be treated as a light-weight container and there is no prohibition to managing multiple Kernel instances.

One of the primary cases where managing multple Kernel instances may be a simplifying factor has to do with plug-in / function definition. Consider an example where several plug-ins are utilized and desired to be partitioned according to role. Perhaps a researcher agent may have access to certain data sources, a analysis agent may have access to evaluation tools, and a manager agent is coordinating the broader operation. Defining all of the plug-ins / functions for each role in a single shared Kernel doesn't align with the role partitioning (i.e. the analysis agent could access the data-sources on its own...but perhaps without sufficient context / instruction).

For this reason, a starting point in agent space, we initially emphasized the affinitization of an agent to its own specific Kernel as we expect cases that rely on plug-ins / function-calling to be dominant:

Coming back to your example, I'd expect you intended to include the kernel assignment:

ChatCompletionAgent agent1 = 
  new()
  {
    Kernel = builder.Build(),
    ...
  };

For now, I'd be curious on your thoughts on an alternate approach (should you need to maintain a uber-Kernel as your source):

Kernel kernel = builder.Build();
ChatCompletionAgent agent1 = 
  new()
  {
    Kernel = CreateKernelWithService(kernel, "ser-b")
    ...
  };
  
Kernel CreateKernelWithService(Kernel kernel, string serviceId)
{
    IKernelBuilder builder = Kernel.CreateBuilder();
    builder.Services.AddSingleton(kernel.Services.GetRequiredKeyedService<IChatCompletionService>(serviceId));
    return builder.Build();
}

strikene

strikene, thank you for this insightful question. We have a variety of cases that warrant the support your are proposing that have high priority for further exploration / support.

As you've identified, the current implementation does not currently utilize service selection by id/key. This results in default service-selection logic in : https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/Kernel.cs#L229`Kernel`

Taking a step back, your insight seems to intersect the perennial question on whether to use as a singleton. There are certainly compelling scenarios to consuming a shared kernel with multiple keyed-services. Ultimately, can be treated as a light-weight container and there is no prohibition to managing multiple instances.Kernel``Kernel``Kernel

One of the primary cases where managing multple instances may be a simplifying factor has to do with plug-in / function definition. Consider an example where several plug-ins are utilized and desired to be partitioned according to role. Perhaps a researcher agent may have access to certain data sources, a analysis agent may have access to evaluation tools, and a manager agent is coordinating the broader operation. Defining all of the plug-ins / functions for each role in a single shared doesn't align with the role partitioning (i.e. the analysis agent could access the data-sources on its own...but perhaps without sufficient context / instruction).Kernel``Kernel

For this reason, a starting point in agent space, we initially emphasized the affinitization of an agent to its own specific as we expect cases that rely on plug-ins / function-calling to be dominant:Kernel

https://github.com/microsoft/semantic-kernel/tree/main/dotnet/samples/GettingStartedWithAgents

https://github.com/microsoft/semantic-kernel/tree/main/dotnet/samples/Concepts/Agents

Coming back to your example, I'd expect you intended to include the kernel assignment:
ChatCompletionAgent agent1 = 
  new()
  {
    Kernel = builder.Build(),
    ...
  };
For now, I'd be curious on your thoughts on an alternate approach (should you need to maintain a uber-Kernel as your source):
Kernel kernel = builder.Build();
ChatCompletionAgent agent1 = 
  new()
  {
    Kernel = CreateKernelWithService(kernel, "ser-b")
    ...
  };
  
Kernel CreateKernelWithService(Kernel kernel, string serviceId)
{
    IKernelBuilder builder = Kernel.CreateBuilder();
    builder.Services.AddSingleton(kernel.Services.GetRequiredKeyedService<IChatCompletionService>(serviceId));
    return builder.Build();
}  

hi

I understand very well that when using proxies, provide separate Kernel for each agent to distinguish the AOAI instances used. If it was a new API, I would definitely do it.
In my case, we will assign a kernel to each user, and all log, azure storage, or aoai access rights in this kernel will be based on the user's identity, and since there will be both GPT4/4O/4T users can choose the model they use for the conversation (although 4 can already be replaced by new models, of course their prices are not the same, and in some cases GPT3 will even be chosen). to evaluate the effect of prompt words under different models.

For the code below, I think it might not be as simple as specifying the serviceid when sharing the kernel (of course this is for the consumer, since he doesn't really 'share' and still creates one)

Kernel kernel = builder.Build();
ChatCompletionAgent agent1 = 
  new()
  {
    Kernel = CreateKernelWithService(kernel, "ser-b")
    ...
  };
  
Kernel CreateKernelWithService(Kernel kernel, string serviceId)
{
    IKernelBuilder builder = Kernel.CreateBuilder();
    builder.Services.AddSingleton(kernel.Services.GetRequiredKeyedService<IChatCompletionService>(serviceId));
    return builder.Build();
}

I admit that the agent is correct about the dominance of functions/plugins, which would be in line with its positioning. Because the plug-in/function/prompt determines the role of the agent, and the model that assumes that role is not unique.

Thank you for the additional detail. I will continue to move forward with determining service selector support and had just desired to provide an option that might unblock you in the meantime.

Thank you for the additional detail. I will continue to move forward with determining service selector support and had just desired to provide an option that might unblock you in the meantime.

Perhaps this feature is not frequently used, but upon hearing this news, I am still grateful for the team's decision. Looking forward to the new version

.Net Agents - `ChatCompletionAgent` Pre-Graduation (#7900)

.Net Agents - `ChatCompletionAgent` Pre-Graduation (microsoft#7900)

Navigation Menu

Repository navigation

.Net Can I specify the Openai service instance used by the Agent? #6523

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

.Net Can I specify the Openai service instance used by the Agent? #6523

Description

Activity

crickman commented on Jun 4, 2024

strikene commented on Jun 4, 2024

crickman commented on Jun 5, 2024

strikene commented on Jun 11, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions