Skip to content

.Net: Pass Kernel to connectors and implement automatic OpenAI function calling - #3754

Merged
Stephen Toub (stephentoub) merged 9 commits into
microsoft:mainfrom
stephentoub:passkernelconnectors
Nov 30, 2023
Merged

Stephen Toub (stephentoub) merged 9 commits into
microsoft:mainfrom
stephentoub:passkernelconnectors

Conversation

@stephentoub

@stephentoub Stephen Toub (stephentoub) commented Nov 29, 2023 •

Copy link
Copy Markdown
Member

Closes #3687
Closes #2932

This PR includes three commits:

Commit 1: Simply updates XML comments for Kernel arguments to standardize them.

Commit 2: Updates the connector interfaces (e.g. IChatCompletion) to accept an optional Kernel argument. It's passed through everywhere, but the actual connector implementations largely ignore it, since they don't need anything from it right now.

Commit 3: Updates the {Azure} OpenAI connector to support automatic function calling, whether those functions are explicitly specified in OpenAIPromptExecutionSettings or are implicitly available via the passed in Kernel. The behavior is opt-in, via a revised setting on the OpenAIPromptExecutionSettings class. Rather than just specifying a string (which had been "auto", "none" or a specific function name), this now exposes a property of a new type FunctionCallBehavior, which can be set to various instances to achieve the desired results:

  • FunctionCallBehavior.EnableKernelFunctions - the functions in the provided kernel will be sent but won't be automatically invoked if requested
  • FunctionCallBehavior.AutoInvokeKernelFunctions - the functions in the provided kernel will be sent and will be automatically invoked if requested
  • FunctionCallBehavior.EnableFunctions(functionsList, autoInvoke) - the functions specified will be sent and whether they'll be automatically invoked is determined by the autoInvoke argument and by whether they're available in the provided kernel
  • FunctionCallBehavior.RequireFunction(function, autoInvoke) - the specified function will be sent to the service as a request to function call it

The auto-invoke behavior is wired up based on whether autoInvoke is true. The FunctionCallBehavior.AutoInvokeKernelFunctions can also be opted-into in a service-agnostic manner by setting a named property in the base PromptExecutionSettings class's dictionary; future service implementations can respect that as well if they either have a built-in notion of function calling (which some other LLMs do) or if we want to simulate it with a planner baked into the connector implementation.

Please review this both for my API choices and for implementation details. Roger Barreto (@rogerbarreto), in particular I'd appreciate it if you could look at how streaming is handled. Gina Triolo (@gitri-ms), I'd appreciate it if you could review the function calling APIs and implementation support.

With all of this, this is now the "Hello, world" of function calling:

Kernel kernel = new KernelBuilder()
    .WithOpenAIChatCompletion("gpt-3.5-turbo-1106", apiKey)
    .ConfigurePlugins(plugins => plugins.AddPluginFromObject<TimePlugin>())
    .Build();

OpenAIPromptExecutionSettings settings = new() { FunctionCallBehavior = FunctionCallBehavior.AutoInvokeKernelFunctions };
Console.WriteLine(await kernel.InvokePromptAsync("What is the current time?", settings));

Now that Kernels are passed into connectors, connectors have access to all the functions stored in the kernel.  This enables the {Azure} OpenAI connectors to automatically invoke supplied functions in response to the service requesting a function call.  The behavior is opt-in, via a revised setting on the OpenAIPromptExecutionSettings class. Rather than just choose a string, this now exposes a property of type FunctionCallBehavior, which can be set to various instances to achieve the desired results:
- FunctionCallBehavior.ProvideKernelFunctions - the functions in the provided kernel will be sent but won't be automatically invoked if requested
- FunctionCallBehavior.AutoInvokeKernelFunctions - the functions in the provided kernel will be sent and will be automatically invoked if requested
- FunctionCallBehavior.ProvideFunctions(functionsList, autoInvoke) - the functions specified will be sent and whether they'll be automatically invoked is determined by the autoInvoke argument and by whether they're available in the provided kernel
- FunctionCallBehavior.RequireFunction(function, autoInvoke) - the specified function will be sent to the service as a request to function call it

The auto-invoke behavior is wired up based on whether autoInvoke is true. The FunctionCallBehavior.AutoInvokeKernelFunctions can also be opted-into in a service-agnostic manner by setting a named property in the base PromptExecutionSettings class; future service implementations can respect that as well if they either have a built-in notion of function calling (which some other LLMs do) or if we want to simulate it with a planner baked into the connector implementation.
@stephentoub
Stephen Toub (stephentoub) requested a review from a team as a code owner November 29, 2023 19:23
@shawncal Shawn Callegari (shawncal) added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel kernel.core labels Nov 29, 2023
@github-actions github-actions Bot changed the title Pass Kernel to connectors and implement automatic OpenAI function calling .Net: Pass Kernel to connectors and implement automatic OpenAI function calling Nov 29, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this, will benefit on many use cases,

This is considerably a new architectural change where connectors can trigger function themselfs, making the LLM the actual plan executor.

@stephentoub
Stephen Toub (stephentoub) added this pull request to the merge queue Nov 30, 2023
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code

Projects

None yet

6 participants