Skip to content

Commit f5baf9e

Browse files
Merge branch 'microsoft:main' into update-modelid
2 parents f76a4b0 + f959256 commit f5baf9e

161 files changed

Lines changed: 3232 additions & 1384 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
---
2+
# These are optional elements. Feel free to remove any of them.
3+
status: accepted
4+
contact: dmytrostruk
5+
date: 2023-01-23
6+
deciders: sergeymenshykh, markwallace, rbarreto, stephentoub, dmytrostruk
7+
---
8+
9+
# Kernel Filters
10+
11+
## Context and Problem Statement
12+
13+
Current way of intercepting some event during function execution works as expected using Kernel Events and event handlers. Example:
14+
15+
```csharp
16+
ILogger logger = loggerFactory.CreateLogger("MyLogger");
17+
18+
var kernel = Kernel.CreateBuilder()
19+
.AddOpenAIChatCompletion(
20+
modelId: TestConfiguration.OpenAI.ChatModelId,
21+
apiKey: TestConfiguration.OpenAI.ApiKey)
22+
.Build();
23+
24+
void MyInvokingHandler(object? sender, FunctionInvokingEventArgs e)
25+
{
26+
logger.LogInformation("Invoking: {FunctionName}", e.Function.Name)
27+
}
28+
29+
void MyInvokedHandler(object? sender, FunctionInvokedEventArgs e)
30+
{
31+
if (e.Result.Metadata is not null && e.Result.Metadata.ContainsKey("Usage"))
32+
{
33+
logger.LogInformation("Token usage: {TokenUsage}", e.Result.Metadata?["Usage"]?.AsJson());
34+
}
35+
}
36+
37+
kernel.FunctionInvoking += MyInvokingHandler;
38+
kernel.FunctionInvoked += MyInvokedHandler;
39+
40+
var result = await kernel.InvokePromptAsync("How many days until Christmas? Explain your thinking.")
41+
```
42+
43+
There are a couple of problems with this approach:
44+
45+
1. Event handlers does not support dependency injection. It's hard to get access to specific service, which is registered in application, unless the handler is defined in the same scope where specific service is available. This approach provides some limitations in what place in solution the handler could be defined. (e.g. If developer wants to use `ILoggerFactory` in handler, the handler should be defined in place where `ILoggerFactory` instance is available).
46+
2. It's not clear in what specific period of application runtime the handler should be attached to kernel. Also, it's not clear if developer needs to detach it at some point.
47+
3. Mechanism of events and event handlers in .NET may not be familiar to .NET developers who didn't work with events previously.
48+
49+
<!-- This is an optional element. Feel free to remove. -->
50+
51+
## Decision Drivers
52+
53+
1. Dependency injection for handlers should be supported to easily access registered services within application.
54+
2. There should not be any limitations where handlers are defined within solution, whether it's Startup.cs or separate file.
55+
3. There should be clear way of registering and removing handlers at specific point of application runtime.
56+
4. The mechanism of receiving and processing events in Kernel should be easy and common in .NET ecosystem.
57+
5. New approach should support the same functionality that is available in Kernel Events - cancel function execution, change kernel arguments, change rendered prompt before sending it to AI etc.
58+
59+
## Decision Outcome
60+
61+
Introduce Kernel Filters - the approach of receiving the events in Kernel in similar way as action filters in ASP.NET.
62+
63+
Two new abstractions will be used across Semantic Kernel and developers will have to implement these abstractions in a way that will cover their needs.
64+
65+
For function-related events: `IFunctionFilter`
66+
67+
```csharp
68+
public interface IFunctionFilter
69+
{
70+
void OnFunctionInvoking(FunctionInvokingContext context);
71+
72+
void OnFunctionInvoked(FunctionInvokedContext context);
73+
}
74+
```
75+
76+
For prompt-related events: `IPromptFilter`
77+
78+
```csharp
79+
public interface IPromptFilter
80+
{
81+
void OnPromptRendering(PromptRenderingContext context);
82+
83+
void OnPromptRendered(PromptRenderedContext context);
84+
}
85+
```
86+
87+
New approach will allow developers to define filters in separate classes and easily inject required services to process kernel event correctly:
88+
89+
MyFunctionFilter.cs - filter with the same logic as event handler presented above:
90+
91+
```csharp
92+
public sealed class MyFunctionFilter : IFunctionFilter
93+
{
94+
private readonly ILogger _logger;
95+
96+
public MyFunctionFilter(ILoggerFactory loggerFactory)
97+
{
98+
this._logger = loggerFactory.CreateLogger("MyLogger");
99+
}
100+
101+
public void OnFunctionInvoking(FunctionInvokingContext context)
102+
{
103+
this._logger.LogInformation("Invoking {FunctionName}", context.Function.Name);
104+
}
105+
106+
public void OnFunctionInvoked(FunctionInvokedContext context)
107+
{
108+
var metadata = context.Result.Metadata;
109+
110+
if (metadata is not null && metadata.ContainsKey("Usage"))
111+
{
112+
this._logger.LogInformation("Token usage: {TokenUsage}", metadata["Usage"]?.AsJson());
113+
}
114+
}
115+
}
116+
```
117+
118+
As soon as new filter is defined, it's easy to configure it to be used in Kernel using dependency injection (pre-construction) or add filter after Kernel initialization (post-construction):
119+
120+
```csharp
121+
IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
122+
kernelBuilder.AddOpenAIChatCompletion(
123+
modelId: TestConfiguration.OpenAI.ChatModelId,
124+
apiKey: TestConfiguration.OpenAI.ApiKey);
125+
126+
// Adding filter with DI (pre-construction)
127+
kernelBuilder.Services.AddSingleton<IFunctionFilter, MyFunctionFilter>();
128+
129+
Kernel kernel = kernelBuilder.Build();
130+
131+
// Adding filter after Kernel initialization (post-construction)
132+
// kernel.FunctionFilters.Add(new MyAwesomeFilter());
133+
134+
var result = await kernel.InvokePromptAsync("How many days until Christmas? Explain your thinking.");
135+
```
136+
137+
It's also possible to configure multiple filters which will be triggered in order of registration:
138+
139+
```csharp
140+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter1>();
141+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter2>();
142+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter3>();
143+
```
144+
145+
And it's possible to change the order of filter execution in runtime or remove specific filter if needed:
146+
147+
```csharp
148+
kernel.FunctionFilters.Insert(0, new InitialFilter());
149+
kernel.FunctionFilters.RemoveAt(1);
150+
```

‎dotnet/Directory.Packages.props‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,15 +51,15 @@
5151
<PackageVersion Include="xretry" Version="1.9.0" />
5252
<PackageVersion Include="coverlet.collector" Version="6.0.0" />
5353
<!-- Plugins -->
54-
<PackageVersion Include="DocumentFormat.OpenXml" Version="3.0.0" />
54+
<PackageVersion Include="DocumentFormat.OpenXml" Version="3.0.1" />
5555
<PackageVersion Include="Microsoft.Data.Sqlite" Version="8.0.0" />
5656
<PackageVersion Include="DuckDB.NET.Data.Full" Version="0.9.2" />
5757
<PackageVersion Include="DuckDB.NET.Data" Version="0.9.2" />
5858
<PackageVersion Include="MongoDB.Driver" Version="2.23.1" />
5959
<PackageVersion Include="Microsoft.Graph" Version="[4.51.0, 5)" />
6060
<PackageVersion Include="Microsoft.Azure.Cosmos" Version="[3.32.3, )" />
6161
<PackageVersion Include="Microsoft.Identity.Client.Extensions.Msal" Version="[2.28.0, )" />
62-
<PackageVersion Include="Microsoft.OpenApi" Version="1.6.11" />
62+
<PackageVersion Include="Microsoft.OpenApi" Version="1.6.12" />
6363
<PackageVersion Include="Microsoft.OpenApi.Readers" Version="1.6.11" />
6464
<PackageVersion Include="Google.Apis.CustomSearchAPI.v1" Version="[1.60.0.3001, )" />
6565
<PackageVersion Include="Grpc.Net.Client" Version="2.60.0" />

‎dotnet/SK-dotnet.sln‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,9 @@ EndProject
116116
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "System", "System", "{3CDE10B2-AE8F-4FC4-8D55-92D4AD32E144}"
117117
ProjectSection(SolutionItems) = preProject
118118
src\InternalUtilities\src\System\EnvExtensions.cs = src\InternalUtilities\src\System\EnvExtensions.cs
119+
src\InternalUtilities\src\System\InternalTypeConverter.cs = src\InternalUtilities\src\System\InternalTypeConverter.cs
120+
src\InternalUtilities\src\System\TypeConverterFactory.cs = src\InternalUtilities\src\System\TypeConverterFactory.cs
121+
src\InternalUtilities\src\System\NonNullCollection.cs = src\InternalUtilities\src\System\NonNullCollection.cs
119122
EndProjectSection
120123
EndProject
121124
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Type", "Type", "{E85EA4D0-BB7E-4DFD-882F-A76EB8C0B8FF}"

‎dotnet/docs/EXPERIMENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ You can use the following diagnostic IDs to ignore warnings or errors for a part
1515
- SKEXP0001: Embedding services
1616
- SKEXP0002: Image services
1717
- SKEXP0003: Memory connectors
18-
- SKEXP0004: Kernel Events
18+
- SKEXP0004: Kernel Filters
1919

2020
## OpenAI and Azure OpenAI services
2121

‎dotnet/nuget/nuget-package.props‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<Project>
22
<PropertyGroup>
33
<!-- Central version prefix - applies to all nuget packages. -->
4-
<VersionPrefix>1.1.0</VersionPrefix>
4+
<VersionPrefix>1.2.0</VersionPrefix>
55

66
<PackageVersion Condition="'$(VersionSuffix)' != ''">$(VersionPrefix)-$(VersionSuffix)</PackageVersion>
77
<PackageVersion Condition="'$(VersionSuffix)' == ''">$(VersionPrefix)</PackageVersion>
@@ -10,7 +10,7 @@
1010
<IsPackable>true</IsPackable>
1111

1212
<!-- Package validation. Baseline Version should be lower than current version. -->
13-
<PackageValidationBaselineVersion>1.1.0</PackageValidationBaselineVersion>
13+
<PackageValidationBaselineVersion>1.2.0</PackageValidationBaselineVersion>
1414
<!-- Validate assembly attributes only for Publish builds -->
1515
<NoWarn Condition="'$(Configuration)' != 'Publish'">$(NoWarn);CP0003</NoWarn>
1616
<!-- Do not validate reference assemblies -->

‎dotnet/samples/KernelSyntaxExamples/BaseTest.cs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright (c) Microsoft. All rights reserved.
22

33
using Microsoft.Extensions.Configuration;
4+
using Microsoft.Extensions.Logging;
45
using RepoUtils;
56
using Xunit.Abstractions;
67

@@ -10,9 +11,13 @@ public abstract class BaseTest
1011
{
1112
protected ITestOutputHelper Output { get; }
1213

14+
protected ILoggerFactory LoggerFactory { get; }
15+
1316
protected BaseTest(ITestOutputHelper output)
1417
{
1518
this.Output = output;
19+
this.LoggerFactory = new XunitLogger<object>(output);
20+
1621
LoadUserSecrets();
1722
}
1823

‎dotnet/samples/KernelSyntaxExamples/Example57_KernelHooks.cs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@
1010

1111
namespace Examples;
1212

13+
#pragma warning disable CS0618 // Events are deprecated
14+
1315
public class Example57_KernelHooks : BaseTest
1416
{
1517
/// <summary>

0 commit comments

Comments
 (0)