Skip to content

Introduce @OnThinking annotation for reasoning output handlers - #2460

Merged
geoand merged 1 commit into
quarkiverse:mainfrom
omatheusmesmo:feat/thinking-annotation-and-cdi-event
May 21, 2026
Merged

geoand merged 1 commit into
quarkiverse:mainfrom
omatheusmesmo:feat/thinking-annotation-and-cdi-event

Conversation

@omatheusmesmo

@omatheusmesmo omatheusmesmo commented May 15, 2026 •

Copy link
Copy Markdown
Collaborator

Describe your changes

Reasoning models return a "thinking" field alongside the final answer, but Quarkus had no way to surface that content to the application. This PR adds a provider-agnostic consumption mechanism for non-streaming AI service responses.

A new @OnThinking annotation marks a static method on the AI service interface as the handler for the model's reasoning output. The handler is invoked whenever the response carries a non-blank AiMessage.thinking(). The return type and signatures of the regular AI service methods do not change.

@RegisterAiService
interface MathAssistant {
    String solve(String problem);

    @OnThinking
    static void onThinking(ThinkingEmitted event) {
        Log.infof("[%s] %s", event.methodName(), event.text());
    }
}

The annotation does not enable thinking on the model itself - that remains each provider's responsibility (e.g. model-options.think=true on Ollama), because langchain4j does not unify the request-side knob across providers.

Chat Scopes integration

For @ChatScoped AI services that want to forward reasoning to the connected client through the existing ChatRouteContext.response().thinking(...) channel, the chat-scopes module ships a one-line helper:

@RegisterAiService
@ChatScoped
interface WeaponBuilder {
    @ChatRoute("weaponBuilder")
    String build(@UserMessage String msg);

    @OnThinking
    static void onThinking(ThinkingEmitted event) {
        ChatRouteThinking.forward(event);
    }
}

Forwarding is opt-in by design: thinking content can be large or sensitive, so the framework never sends it to clients implicitly.


@omatheusmesmo
omatheusmesmo requested a review from a team as a code owner May 15, 2026 18:51
@geoand

geoand commented May 17, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks a lot for this, but TBH, the use of CDI events for this is weird.

IMHO, a more intuitive API would look something like:

@RegisterAiService
interface MathAssistant {

    String solve(String problem);
 
    @Thinking
    static void onThinking(ThinkingEmitted event) {
        ...
    }
}

@quarkus-bot

This comment has been minimized.

@patriot1burke

Copy link
Copy Markdown
Collaborator

Would love to have this Chat scopes aware too.

@omatheusmesmo

Copy link
Copy Markdown
Collaborator Author

Thanks a lot for this, but TBH, the use of CDI events for this is weird.

IMHO, a more intuitive API would look something like:

@RegisterAiService
interface MathAssistant {

    String solve(String problem);
 
    @Thinking
    static void onThinking(ThinkingEmitted event) {
        ...
    }
}

I'll work on it, thanks for pointing out.

@omatheusmesmo

Copy link
Copy Markdown
Collaborator Author

Would love to have this Chat scopes aware too.

I'll take a look into it too, thanks.

@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from 2e72508 to 6aed954 Compare May 17, 2026 20:27
@omatheusmesmo omatheusmesmo changed the title Introduce @Thinking annotation and ThinkingEmitted CDI event Introduce @Thinking annotation for reasoning output handlers May 17, 2026
@omatheusmesmo

omatheusmesmo commented May 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Hello @geoand @patriot1burke , hope you are doing well.

I updated my PR, could you kindly take another look? I hope now is adequate to what you both expected.

I appreciate your review, thanks.

@quarkus-bot

This comment has been minimized.

@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from 6aed954 to aced97c Compare May 18, 2026 10:43
@omatheusmesmo

Copy link
Copy Markdown
Collaborator Author

Hi @geoand , thanks, I updated the PR.

@quarkus-bot

This comment has been minimized.

@geoand geoand left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice! I just added one comment, but I also need @patriot1burke to review the chat scopes part

@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from aced97c to d640305 Compare May 18, 2026 15:46
@omatheusmesmo
omatheusmesmo requested a review from geoand May 18, 2026 15:48
@quarkus-bot

This comment has been minimized.

@geoand
geoand requested a review from patriot1burke May 19, 2026 05:51
@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from d640305 to 4aaddac Compare May 19, 2026 09:09
@omatheusmesmo omatheusmesmo changed the title Introduce @Thinking annotation for reasoning output handlers Introduce @OnThinking annotation for reasoning output handlers May 19, 2026
@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from 4aaddac to 551281d Compare May 19, 2026 09:13

@geoand geoand left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@patriot1burke can you also check?

@quarkus-bot

This comment has been minimized.

@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from 551281d to 8f506b2 Compare May 19, 2026 10:17
@quarkus-bot

This comment has been minimized.

Adds a provider-agnostic way to consume reasoning/thinking output from
non-streaming AI service responses. The annotation marks a static method
on the AI service interface that the runtime invokes whenever the
response carries non-blank AiMessage.thinking() text. Enabling thinking
on the model is still the responsibility of each provider (for example
returnThinking on the OpenAI chat model builder, or model-options.think
on Ollama), since langchain4j does not unify the request-side knob
across providers.

In the chat-scopes module, a new ChatRouteThinking helper exposes a
one-line bridge to the existing ResponseChannel.thinking() so a handler
on a `@ChatScoped` service can forward reasoning to the chat client
without depending on CDI lookup directly. The forwarding is opt-in so
that thinking content is never sent to clients by accident.

* Fix quarkiverse#1685

Signed-off-by: Matheus Oliveira <[email protected]>
@omatheusmesmo
omatheusmesmo force-pushed the feat/thinking-annotation-and-cdi-event branch from 8f506b2 to a3ed8b5 Compare May 20, 2026 12:40
@quarkus-bot

quarkus-bot Bot commented May 20, 2026

Copy link
Copy Markdown

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit a3ed8b5.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

@geoand
geoand merged commit 42d1f31 into quarkiverse:main May 21, 2026
90 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Introduce configuration option for setting the thinking option in Ollama

3 participants