Repository navigation
Conversation
|
Thanks a lot for this, but TBH, the use of CDI events for this is weird. IMHO, a more intuitive API would look something like: @RegisterAiService
interface MathAssistant {
String solve(String problem);
@Thinking
static void onThinking(ThinkingEmitted event) {
...
}
} |
This comment has been minimized.
This comment has been minimized.
|
Would love to have this Chat scopes aware too. |
I'll work on it, thanks for pointing out. |
I'll take a look into it too, thanks. |
2e72508 to
6aed954
Compare
|
Hello @geoand @patriot1burke , hope you are doing well. I updated my PR, could you kindly take another look? I hope now is adequate to what you both expected. I appreciate your review, thanks. |
This comment has been minimized.
This comment has been minimized.
6aed954 to
aced97c
Compare
|
Hi @geoand , thanks, I updated the PR. |
This comment has been minimized.
This comment has been minimized.
geoand
left a comment
There was a problem hiding this comment.
Looks nice! I just added one comment, but I also need @patriot1burke to review the chat scopes part
aced97c to
d640305
Compare
This comment has been minimized.
This comment has been minimized.
d640305 to
4aaddac
Compare
4aaddac to
551281d
Compare
geoand
left a comment
There was a problem hiding this comment.
Thanks a lot!
@patriot1burke can you also check?
This comment has been minimized.
This comment has been minimized.
551281d to
8f506b2
Compare
This comment has been minimized.
This comment has been minimized.
Adds a provider-agnostic way to consume reasoning/thinking output from non-streaming AI service responses. The annotation marks a static method on the AI service interface that the runtime invokes whenever the response carries non-blank AiMessage.thinking() text. Enabling thinking on the model is still the responsibility of each provider (for example returnThinking on the OpenAI chat model builder, or model-options.think on Ollama), since langchain4j does not unify the request-side knob across providers. In the chat-scopes module, a new ChatRouteThinking helper exposes a one-line bridge to the existing ResponseChannel.thinking() so a handler on a `@ChatScoped` service can forward reasoning to the chat client without depending on CDI lookup directly. The forwarding is opt-in so that thinking content is never sent to clients by accident. * Fix quarkiverse#1685 Signed-off-by: Matheus Oliveira <[email protected]>
8f506b2 to
a3ed8b5
Compare
Status for workflow
|
Describe your changes
Reasoning models return a "thinking" field alongside the final answer, but Quarkus had no way to surface that content to the application. This PR adds a provider-agnostic consumption mechanism for non-streaming AI service responses.
A new
@OnThinkingannotation marks astaticmethod on the AI service interface as the handler for the model's reasoning output. The handler is invoked whenever the response carries a non-blankAiMessage.thinking(). The return type and signatures of the regular AI service methods do not change.The annotation does not enable thinking on the model itself - that remains each provider's responsibility (e.g.
model-options.think=trueon Ollama), because langchain4j does not unify the request-side knob across providers.Chat Scopes integration
For
@ChatScopedAI services that want to forward reasoning to the connected client through the existingChatRouteContext.response().thinking(...)channel, thechat-scopesmodule ships a one-line helper:Forwarding is opt-in by design: thinking content can be large or sensitive, so the framework never sends it to clients implicitly.
thinkingoption in Ollama #1685