Repository navigation
.Net: Function calling: Update ChatMessage to support 'name' #3017
Description
Important
Labeled Urgent because it may require a breaking change. Please reduce the priority if this is not the case.
To fully support the function calling RAG pattern, we need to have a "name" field in our chat message objects. However, there are concerns about putting OpenAI specific fields in abstraction classes, so this may require a refactor of the chat message abstraction.
Here's an example of adding the "name" field to enable the full function calling scenario: gitri-ms#2
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Update ChatMessage to supprt 'name' and 'function_calling'[/-][+]Function calling: Update ChatMessage to supprt 'name' and 'function_calling'[/+][-]Function calling: Update ChatMessage to supprt 'name' and 'function_calling'[/-][+]Function calling: Update ChatMessage to support 'name' and 'function_calling'[/+][-]Function calling: Update ChatMessage to support 'name' and 'function_calling'[/-][+].Net: Function calling: Update ChatMessage to support 'name' and 'function_calling'[/+]stephentoub commentedon Oct 12, 2023
If we're not comfortable adding a name property to a chat message in support of this, we could also just add a dictionary to the message so that additional context can be provided and interpreted by the chat completion service however it likes:
main...stephentoub:semantic-kernel:fixfunctioncalling
We need to do something here, whatever approach we pick. As it stands, the feature that's merged isn't actually usable as-is, since there's no way to send the function result back to the LLM.
madsbolaris commentedon Oct 12, 2023
Our min bar for getting function calling done for v1.0.0 is to make sure that it can be used by Stepwise planner. Need to create a POC of stepwise planner leveraging function calling as part of this item.
stephentoub commentedon Oct 16, 2023
@matthewbolanos, I actually think this is the min bar:
#2932 (comment)
Exposing a planner around OpenAI function calling is nice to have. But the bread-and-butter here is enabling a developer to do the normal chat completion flow while also factoring in functions. We can make that so seamless, and not doing so is a huge missed opportunity. Doing it only as part of a planner means that someone implementing chat needs to proactively use a planner as part of each chat message in order to proactively populate the chat message with the additional context, just in case it might possibly be needed. By letting OpenAI function calling handle it, not only is the consumption simpler (via surface area like what I've proposed), it's also more efficient, as you only need to do the planning if it's actually required and only when it's actually required.
leungkimming commentedon Oct 19, 2023
I agree with Stephen. In my use case, my prompt will trigger multiple functions. I need to return the function results to LLM for it to compile the final reply consisting of all the function call results. I don't want to fallback to native OpenAI API calls.
leungkimming commentedon Oct 23, 2023
Gina Triolo (@gitri-ms), I have tested your solution and found an issue. I have a CheckStock function and my prompt is "How many tables and chairs are there in stock?". Note that LLM should issue 2 function calling messages (I have added a do-while loop in Example59). However, with your version, LLM only call CheckStock once. In my original working version using native OpenAI, the Messages in the ChatCompletionsOptions keeps both the function calling assistance message and the function return user message. In your solution, only function return user
messages are kept in the abstracted ChatMessageBase. Hence, in ClientBase.cs, when CreateChatCompletionsOptions populate ChatCompletionsOptions using the abstracted ChatMessageBase, there are no function calling assistance message histories passing back to the LLM.
Since this enhancement is already in the in-progress sprint, please take this into consideration.
[-].Net: Function calling: Update ChatMessage to support 'name' and 'function_calling'[/-][+].Net: Function calling: Update ChatMessage to support 'name'[/+]markwallace-microsoft commentedon Oct 31, 2023
PR #3302
.Net: Function calling: Add function author role and name property (#…