@microsoft-github-policy-service agree company="Klippa App B.V."

looks like the build is failing, could you also run ruff check and ruff format in your next commit?

looks like the build is failing, could you also run ruff check and ruff format in your next commit?

Yeah I wanted to setup pre-commit, but I couldn't get it to install, had some issues with the dotnet tool nbgv.

I like the approach here. Soemthings I think we should do:

1. Emit should be a no-op if the stream is cancelled.
2. We potentially need a way to short-circuit the AI generation if emit is short-circuiting. One approach is to throw on emit if stream is cancelled. But then we'd break all existing handlers. Another approach is for the handlers to have access to this signal, and choose to short-circuit themselves.

I'm leaning toward throwing on emit.

Thoughts?

• Emit should be a no-op if the stream is cancelled.

What do you mean? It already is.

• We potentially need a way to short-circuit the AI generation if emit is short-circuiting. One approach is to throw on emit if stream is cancelled. But then we'd break all existing handlers. Another approach is for the handlers to have access to this signal, and choose to short-circuit themselves.

I'd like the second approach more. Perhaps something using yield? (I'm not really a Python dev so not sure)

Edit:
Nvm, I see there is some EventEmitter system in place already, better to use that.

• Emit should be a no-op if the stream is cancelled.

What do you mean? It already is.

• We potentially need a way to short-circuit the AI generation if emit is short-circuiting. One approach is to throw on emit if stream is cancelled. But then we'd break all existing handlers. Another approach is for the handlers to have access to this signal, and choose to short-circuit themselves.

I'd like the second approach more. Perhaps something using yield? (I'm not really a Python dev so not sure)

Edit: Nvm, I see there is some EventEmitter system in place already, better to use that.

Oops you're right. Ignore 1st.

@jerbob92 why do you prefer second? I think your prompt.send throwing if the stream gets cancelled is ok.

• Emit should be a no-op if the stream is cancelled.

What do you mean? It already is.

• We potentially need a way to short-circuit the AI generation if emit is short-circuiting. One approach is to throw on emit if stream is cancelled. But then we'd break all existing handlers. Another approach is for the handlers to have access to this signal, and choose to short-circuit themselves.

I'd like the second approach more. Perhaps something using yield? (I'm not really a Python dev so not sure)
Edit: Nvm, I see there is some EventEmitter system in place already, better to use that.

Oops you're right. Ignore 1st.

@jerbob92 why do you prefer second? I think your prompt.send throwing if the stream gets cancelled is ok.

Personally I think an event based approach is cleaner, but maybe that would only really make sense if the cancelation may happen without the user doing emits, for that to work the Streamer would either need an endpoint to check for cancelation every x duration or it needs to be an actual HTTP stream. (or maybe I just didn't really get your comment and you're talking about something that's not directly related to the MR)

I like the approach here. Soemthings I think we should do:

1. Emit should be a no-op if the stream is cancelled.
2. We potentially need a way to short-circuit the AI generation if emit is short-circuiting. One approach is to throw on emit if stream is cancelled. But then we'd break all existing handlers. Another approach is for the handlers to have access to this signal, and choose to short-circuit themselves.

I'm leaning toward throwing on emit.

Thoughts?

+1 with Aamir — throwing on emit seems like the right approach. I think event-based only adds value if cancellation happens between emits

Yeah event emitter is less useful here because it would catch the errors asynchronously (out of band), so coordinating it back with where you're emitting in application code will be difficult.

Updated the PR with some changes:

• Added StreamCancelledError (extends asyncio.CancelledError) — a dedicated exception for stream cancellation. emit() now raises this instead of silently returning, which breaks the caller's AI generation loop (e.g. stops the OpenAI stream mid-generation).
• When the handler raises StreamCancelledError, the activity processor catches it and returns a 200 to Teams, preventing retries.
• Fixed close() hanging for up to 30s after cancellation — it now returns immediately when the stream is cancelled.

Tested by:

1. Starting a stream
2. Stopping it
3. Killing the service — no delay (before, there was a ~30s delay since emitting was still silently happening and close() was waiting for the queue to drain)

Then:

1. Starting a stream
2. Stopping it
3. Sending another message — sent normally, no issues running the handler (before, the stream request would get retried because the exception bubbled up as a 500)