Repository navigation
.Net Show how to limit total token usage using filters when using function calling #5145
Description
Describe the bug
A clear and concise description of what the bug is.
User specify MaxTokens in prompt execution settings, which is passed to IChatCompletionService.
OpenAI with enabled auto invoke function may do many request to api, but always is passed same MaxToken value.
So at the end service can use more tokens totally than user specified.
Expected behavior
A clear and concise description of what you expected to happen.
Decrease MaxTokens by used tokens value after each iteration in ChatCompletionService.
Platform
- Language: C#
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
[-].Net MaxTokens in OpenAI function calling[/-][+].Net Show how to set max tokens in OpenAI function calling with filters[/+][-].Net Show how to set max tokens in OpenAI function calling with filters[/-][+].Net Show how to limit total token usage using filters when using function calling[/+].Net: Added an example to show how to limit token usage with filters (#…