Skip to content

.Net: Added an example to show how to limit token usage with filters - #9451

Merged
Dmytro Struk (dmytrostruk) merged 9 commits into
microsoft:mainfrom
dmytrostruk:filters-max-tokens-example
Nov 1, 2024
Merged

Dmytro Struk (dmytrostruk) merged 9 commits into
microsoft:mainfrom
dmytrostruk:filters-max-tokens-example

Conversation

@dmytrostruk

@dmytrostruk Dmytro Struk (dmytrostruk) commented Oct 28, 2024 •

Copy link
Copy Markdown
Member

Motivation and Context

Resolves: #5145

Property OpenAIPromptExecutionSettings.MaxTokens allows specifying maximum number of tokens to generate in one response. In Semantic Kernel, auto function calling may perform multiple requests to AI model, but with the same max tokens value. For example, in case when max tokens = 50, and 3 functions are expected to be called with 3 separate requests to AI model, each request will have max tokens = 50, which in total will result in more tokens used. This example shows how to limit token usage with OpenAIPromptExecutionSettings.MaxTokens property and filter for all requests in the same auto function calling process.

Contribution Checklist

@markwallace-microsoft Mark Wallace (markwallace-microsoft) added the kernel Issues or pull requests impacting the core kernel label Oct 31, 2024
@dmytrostruk
Dmytro Struk (dmytrostruk) added this pull request to the merge queue Nov 1, 2024
Merged via the queue into microsoft:main with commit db45cfa Nov 1, 2024
@dmytrostruk
Dmytro Struk (dmytrostruk) deleted the filters-max-tokens-example branch November 1, 2024 19:22

This branch was previously deployed

1 inactive deployment
integration — a14bd8ce Deployed Nov 1, 2024 by dmytrostruk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation .NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.Net Show how to limit total token usage using filters when using function calling

3 participants