Skip to content

Commit 78f6124

Browse files
authored
Python: close native generators when stream invocations end (#14701)
### Motivation and Context Closing a native function's streaming invocation after its first item does not promptly close the underlying generator. With a retained reference to that generator, its `finally` block remains suspended after the invocation ends. The same occurs when `CancelledError` is injected into the suspended invocation. A plugin that releases resources in `finally` therefore keeps them live beyond the closed invocation. This reproduces through both `KernelFunction.invoke_stream()` and `Kernel.invoke_stream()` with ordinary decorated sync/async native functions. ### Description Use `closing`/`aclosing` around actual Python generators consumed by `KernelFunction.invoke_stream`, and close the newly created function-invocation stream owned by `Kernel.invoke_stream`. Ordinary returned values and custom close-bearing iterable objects retain their current behavior. Public signatures are unchanged. ### Validation Base: `96c9e860db4d7c6da40652d9b421a31b05e98692`. - Added 18 cases retaining the source generators so garbage collection cannot make cleanup appear correct. They cover direct/Kernel entry, sync/async sources, early `aclose`, injected cancellation, normal exhaustion, original exception identity and an ordinary close-bearing list that must remain untouched. - Identical function/kernel suites against an asserted original-source package root: **8 failed, 148 passed, 4 warnings**. Candidate: **156 passed, 4 warnings**. Both runs block socket connections; the existing/dependency warnings are unchanged. - Independent AI review reran those suites and **33 mocked prompt tests**, plus 10 retained-source checks for real task cancellation, awaited async finalizers and cleanup-error priority. - Ruff lint/format checks on the three changed files and `git diff --check`: passed. Test environment: Python 3.13, local editable source and OpenAI 2.26.0 selected from the repository lockfile; not a complete locked all-connectors environment. Full SDK/integration/provider suites were not run. Scope: the native-generator invocation path. Consumers still need to close a retained stream when leaving their own iteration body, for example with `aclosing`. The separate `invoke_prompt_stream` forwarding layer and deeper provider wrappers are outside this fix; no general provider-resource guarantee is claimed. ### Contribution Checklist - [x] Added failing-first regression and compatibility coverage. - [x] Followed contribution guidance and the existing Python source style. - [x] Ran focused local checks with external sockets blocked. - [ ] Full SDK unit/integration matrix; the executed suites and unchanged warnings are listed above. AI disclosure: Codex implemented and tested this change, with an independent AI review, as part of an autonomous upstream contribution benchmark. No personal human code review is claimed.
1 parent 703eb5f commit 78f6124

3 files changed

Lines changed: 100 additions & 16 deletions

File tree

‎python/semantic_kernel/functions/kernel_function.py‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import time
66
from abc import abstractmethod
77
from collections.abc import AsyncGenerator, Callable, Mapping, Sequence
8+
from contextlib import aclosing, closing
89
from copy import copy, deepcopy
910
from inspect import isasyncgen, isgenerator
1011
from typing import TYPE_CHECKING, Any
@@ -352,13 +353,15 @@ async def invoke_stream(
352353
function_results: list[Any] = []
353354
if function_context.result is not None:
354355
if isasyncgen(function_context.result.value):
355-
async for partial in function_context.result.value:
356-
function_results.append(partial)
357-
yield partial
356+
async with aclosing(function_context.result.value) as async_stream:
357+
async for partial in async_stream:
358+
function_results.append(partial)
359+
yield partial
358360
elif isgenerator(function_context.result.value):
359-
for partial in function_context.result.value:
360-
function_results.append(partial)
361-
yield partial
361+
with closing(function_context.result.value) as stream:
362+
for partial in stream:
363+
function_results.append(partial)
364+
yield partial
362365
else:
363366
function_results.append(function_context.result.value)
364367
yield function_context.result

‎python/semantic_kernel/kernel.py‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
import logging
44
from collections.abc import AsyncGenerator, AsyncIterable, Callable
5-
from contextlib import AbstractAsyncContextManager
5+
from contextlib import AbstractAsyncContextManager, aclosing
66
from copy import copy, deepcopy
77
from typing import TYPE_CHECKING, Any, Literal, TypeVar
88

@@ -142,15 +142,16 @@ async def invoke_stream(
142142

143143
function_result: list[list["StreamingContentMixin"] | Any] = []
144144

145-
async for stream_message in function.invoke_stream(self, arguments):
146-
if isinstance(stream_message, FunctionResult) and (
147-
exception := stream_message.metadata.get(METADATA_EXCEPTION_KEY, None)
148-
):
149-
raise KernelInvokeException(
150-
f"Error occurred while invoking function: '{function.fully_qualified_name}'"
151-
) from exception
152-
function_result.append(stream_message)
153-
yield stream_message
145+
async with aclosing(function.invoke_stream(self, arguments)) as stream:
146+
async for stream_message in stream:
147+
if isinstance(stream_message, FunctionResult) and (
148+
exception := stream_message.metadata.get(METADATA_EXCEPTION_KEY, None)
149+
):
150+
raise KernelInvokeException(
151+
f"Error occurred while invoking function: '{function.fully_qualified_name}'"
152+
) from exception
153+
function_result.append(stream_message)
154+
yield stream_message
154155

155156
if return_function_results:
156157
output_function_result: list["StreamingContentMixin"] = []

‎python/tests/unit/functions/test_kernel_function_from_method.py‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Copyright (c) Microsoft. All rights reserved.
22
import logging
3+
from asyncio import CancelledError
34
from collections.abc import AsyncGenerator, Iterable
45
from contextlib import nullcontext
56
from dataclasses import dataclass
@@ -26,6 +27,85 @@
2627
kernel_function_module = import_module("semantic_kernel.functions.kernel_function")
2728

2829

30+
@pytest.mark.parametrize("asynchronous", [False, True])
31+
@pytest.mark.parametrize("via_kernel", [False, True])
32+
@pytest.mark.parametrize("termination", ["close", "cancel", "exhaust", "error"])
33+
async def test_invoke_stream_closes_native_generator(
34+
kernel: Kernel, asynchronous: bool, via_kernel: bool, termination: str
35+
):
36+
closed = []
37+
failure = ValueError("native stream failed")
38+
39+
@kernel_function
40+
def sync_stream() -> Iterable[str]:
41+
try:
42+
yield "first"
43+
if termination == "error":
44+
raise failure
45+
yield "second"
46+
finally:
47+
closed.append(True)
48+
49+
@kernel_function
50+
async def async_stream() -> AsyncGenerator[str, None]:
51+
try:
52+
yield "first"
53+
if termination == "error":
54+
raise failure
55+
yield "second"
56+
finally:
57+
closed.append(True)
58+
59+
# Retain the source so garbage collection cannot stand in for explicit cleanup.
60+
method = async_stream if asynchronous else sync_stream
61+
source = method()
62+
function = KernelFunctionFromMethod(method=method, stream_method=lambda: source)
63+
stream = kernel.invoke_stream(function) if via_kernel else function.invoke_stream(kernel)
64+
try:
65+
assert await anext(stream) == "first"
66+
assert not closed
67+
if termination == "close":
68+
await stream.aclose()
69+
elif termination == "cancel":
70+
with pytest.raises(CancelledError):
71+
await stream.athrow(CancelledError())
72+
elif termination == "error":
73+
with pytest.raises(ValueError) as exc_info:
74+
await anext(stream)
75+
assert exc_info.value is failure
76+
else:
77+
assert [part async for part in stream] == ["second"]
78+
assert closed == [True]
79+
finally:
80+
await stream.aclose()
81+
if asynchronous:
82+
await source.aclose()
83+
else:
84+
source.close()
85+
86+
87+
@pytest.mark.parametrize("via_kernel", [False, True])
88+
async def test_invoke_stream_does_not_close_ordinary_result(kernel: Kernel, via_kernel: bool):
89+
closed = []
90+
91+
class ResultList(list):
92+
def close(self):
93+
closed.append(True)
94+
95+
@kernel_function
96+
def native() -> str:
97+
return "non-streaming result"
98+
99+
result = ResultList(["ordinary result"])
100+
function = KernelFunctionFromMethod(method=native, stream_method=lambda: result)
101+
stream = kernel.invoke_stream(function) if via_kernel else function.invoke_stream(kernel)
102+
parts = [part async for part in stream]
103+
assert len(parts) == 1
104+
assert parts[0].value is result
105+
assert result == ["ordinary result"]
106+
assert not closed
107+
108+
29109
class CustomType(KernelBaseModel):
30110
id: str
31111
name: str

0 commit comments

Comments
 (0)