Repository navigation
.Net: Pass Kernel to connectors and implement automatic OpenAI function calling - #3754
Merged
Stephen Toub (stephentoub) merged 9 commits intoNov 30, 2023
Merged
Conversation
Now that Kernels are passed into connectors, connectors have access to all the functions stored in the kernel. This enables the {Azure} OpenAI connectors to automatically invoke supplied functions in response to the service requesting a function call. The behavior is opt-in, via a revised setting on the OpenAIPromptExecutionSettings class. Rather than just choose a string, this now exposes a property of type FunctionCallBehavior, which can be set to various instances to achieve the desired results:
- FunctionCallBehavior.ProvideKernelFunctions - the functions in the provided kernel will be sent but won't be automatically invoked if requested
- FunctionCallBehavior.AutoInvokeKernelFunctions - the functions in the provided kernel will be sent and will be automatically invoked if requested
- FunctionCallBehavior.ProvideFunctions(functionsList, autoInvoke) - the functions specified will be sent and whether they'll be automatically invoked is determined by the autoInvoke argument and by whether they're available in the provided kernel
- FunctionCallBehavior.RequireFunction(function, autoInvoke) - the specified function will be sent to the service as a request to function call it
The auto-invoke behavior is wired up based on whether autoInvoke is true. The FunctionCallBehavior.AutoInvokeKernelFunctions can also be opted-into in a service-agnostic manner by setting a named property in the base PromptExecutionSettings class; future service implementations can respect that as well if they either have a built-in notion of function calling (which some other LLMs do) or if we want to simulate it with a planner baked into the connector implementation.
Mark Wallace (markwallace-microsoft)
approved these changes
Nov 29, 2023
Roger Barreto (rogerbarreto)
approved these changes
Nov 29, 2023
Roger Barreto (rogerbarreto)
left a comment
Member
There was a problem hiding this comment.
Thanks for adding this, will benefit on many use cases,
This is considerably a new architectural change where connectors can trigger function themselfs, making the LLM the actual plan executor.
Mads Bolaris (madsbolaris)
approved these changes
Nov 30, 2023
Stephen Toub (stephentoub)
force-pushed
the
passkernelconnectors
branch
from
November 30, 2023 02:03
e08aae8 to
0f4093b
Compare
Stephen Toub (stephentoub)
force-pushed
the
passkernelconnectors
branch
from
November 30, 2023 02:27
0f4093b to
23ee978
Compare
Stephen Toub (stephentoub)
enabled auto-merge
November 30, 2023 02:32
Stephen Toub (stephentoub)
disabled auto-merge
November 30, 2023 15:52
SergeyMenshykh
approved these changes
Nov 30, 2023
Stephen Toub (stephentoub)
enabled auto-merge
November 30, 2023 16:30
SergeyMenshykh
approved these changes
Nov 30, 2023
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 30, 2023
Stephen Toub (stephentoub)
enabled auto-merge
November 30, 2023 17:03
SergeyMenshykh
approved these changes
Nov 30, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3687
Closes #2932
This PR includes three commits:
Commit 1: Simply updates XML comments for Kernel arguments to standardize them.
Commit 2: Updates the connector interfaces (e.g. IChatCompletion) to accept an optional Kernel argument. It's passed through everywhere, but the actual connector implementations largely ignore it, since they don't need anything from it right now.
Commit 3: Updates the {Azure} OpenAI connector to support automatic function calling, whether those functions are explicitly specified in OpenAIPromptExecutionSettings or are implicitly available via the passed in Kernel. The behavior is opt-in, via a revised setting on the OpenAIPromptExecutionSettings class. Rather than just specifying a string (which had been "auto", "none" or a specific function name), this now exposes a property of a new type FunctionCallBehavior, which can be set to various instances to achieve the desired results:
The auto-invoke behavior is wired up based on whether autoInvoke is true. The FunctionCallBehavior.AutoInvokeKernelFunctions can also be opted-into in a service-agnostic manner by setting a named property in the base PromptExecutionSettings class's dictionary; future service implementations can respect that as well if they either have a built-in notion of function calling (which some other LLMs do) or if we want to simulate it with a planner baked into the connector implementation.
Please review this both for my API choices and for implementation details. Roger Barreto (@rogerbarreto), in particular I'd appreciate it if you could look at how streaming is handled. Gina Triolo (@gitri-ms), I'd appreciate it if you could review the function calling APIs and implementation support.
With all of this, this is now the "Hello, world" of function calling: