Configurable brokerService for console queries by aho135 · Pull Request #19069 · apache/druid

aho135 · 2026-02-28T18:47:56Z

Description

When using the manual routing strategy in combination with druid.broker.segment.watchedTiers we are losing console visibility for Historicals. This can happen with the combination of configurations

Router

druid.router.sql.enable: true
druid.router.strategies: [{"type":"manual","defaultManualBrokerService":"druid/broker1"}]
druid.router.tierToBrokerMap: {"_default_tier": "druid/broker1", "broker2": "druid/broker2"}

broker1

druid.broker.segment.watchedTiers=["t1","t2"]

broker2

druid.broker.segment.watchedTiers=["t1","t2","t3","t4"]

Since the Router query goes to druid/broker1 which has limited segment visibility, the console will incorrectly report Historicals in t3/t4 as having no data. This patch allows us to configure Router console queries to go to druid/broker2 instead so that the console can have full visibility over Historicals.

Release note

Allows operators to configure brokerService for Router console queries. This can be accomplished by setting something like

consoleBrokerService: 'druid/broker2',

in web-console/console-config.js

Key changed/added classes in this PR

druid-query.ts
entry.tsx

This PR has:

been self-reviewed.
added documentation for new or modified features or behaviors.
a release note entry in the PR description.
added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
been tested in a test Druid cluster.

abhishek-chouhan · 2026-03-02T17:51:06Z

There is also the case where broker1 watches ["t1","t2"] and broker2 watches ["t3","t4"]. In this case, to get a complete view both need to be queried. Would this case work as well?

aho135 · 2026-03-02T20:37:14Z

There is also the case where broker1 watches ["t1","t2"] and broker2 watches ["t3","t4"]. In this case, to get a complete view both need to be queried. Would this case work as well?

Thanks for the review @abhishek-chouhan! This case is not covered. This would be more invasive as it requires merging BrokerServerView from broker1 and broker2. In this scenario I think it would be simpler to spin up an additional broker (e.g. druid/broker-admin) which watches all tiers and configure consoleBrokerService to point there.

abhishekrb19 · 2026-03-02T23:08:52Z

+
+  // Broker service name to use for all console SQL queries (for tier isolation)
+  // This injects "brokerService" into the query context for routing via the manual strategy
+  consoleBrokerService?: string;


What do you think about exposing this config via web-console-config.tsx as well? It has this dialog on the console, which seems cool:

I think web-console-config.tsx is session-based that stores config overrides in the browser, while the ConsoleConfig from console-config.js is persistent via the build. Not super familiar with the web-console code here, so correct me if I'm missing something here.

Can console-config.js be overridden per environment, given that this file is baked into the build as part of the source code?

Thanks for the review @abhishekrb19!

Yeah the console-config.js is persistent via the build, so overriding per environment is not possible. For our own use case this would still be fine, since it would fall back to the default druid/broker if the configured brokerService doesn't exist. Let me take a look at web-console-config.tsx

I like this idea of adding it to the web-console-config. This uses the browser's local storage and persists through restarts

abhishekrb19 · 2026-03-02T23:16:49Z

+        }
+      : sqlQueryPayload;
+
+    sqlResultResp = await Api.instance.post('/druid/v2/sql', payload, { signal });


I see that https://github.com/apache/druid/blob/master/web-console/src/views/explore-view/explore-view.tsx#L64
directly calls the /druid/v2/sql endpoint without invoking queryDruidSql. Should we update it to use this function instead, so the context is appended for the Explore view as well?

Since the Explore tab is more for user-issued queries I thought to leave it out for now. I wanted to limit the scope of this to just Router console queries

Ah I see, thanks for clarifying. I got tripped up by the name consoleBrokerService - I assumed all queries originating from the console would use this, but it sounds like only the console-generated system queries would use it.

What do you think about renaming this to something less ambiguous, like consoleSystemQueryBrokerService? Alternatively, we could leave the name as-is if we expect the scope to expand in the future and instead clarify it in the help text and documentation along with the limitation that @abhishek-chouhan mentioned with watch tiers (this can be done separately in a follow up PR)

Thanks @abhishekrb19 this makes sense. Let me update the name to make clear that it is only for system queries

abhishek-chouhan · 2026-03-02T23:24:26Z

Thanks for the review @abhishek-chouhan! This case is not covered. This would be more invasive as it requires merging BrokerServerView from broker1 and broker2. In this scenario I think it would be simpler to spin up an additional broker (e.g. druid/broker-admin) which watches all tiers and configure consoleBrokerService to point there.

I do think we need the merging functionality to support the watchedTiers functionality in its entirety. While the workaround with a separate broker works, its too heavy of a workaround just to make the console work with watchedTiers.
This change is still valuable, we can take up the merging brokerServerView change as a followup enhancement.

abhishekrb19 · 2026-03-02T23:32:01Z

Since brokerService is part of the query context, would setting baseQueryContext or defaultQueryContext in console-config.js work out of the box?

These settings only apply to queries run from the Query tab and not the console queries

…config

…vice

abhishekrb19

Lgtm, thanks @aho135!

Configurable brokerService for console queries

5a4fee0

github-actions Bot added the Area - Web Console label Feb 28, 2026

aho135 requested a review from vogievetsky February 28, 2026 18:49

abhishekrb19 reviewed Mar 2, 2026

View reviewed changes

aho135 added 2 commits March 3, 2026 20:57

Add support to set brokerService for console queries via web-console-…

82ae0d4

…config

npm run prettify

51b26cf

abhishekrb19 reviewed Mar 4, 2026

View reviewed changes

Comment thread web-console/src/utils/druid-query.ts Outdated

aho135 added 2 commits March 4, 2026 13:29

Remove brokerService for queryDruidRune

b7a3f05

Update from setConsoleBrokerService to setConsoleSystemQueryBrokerSer…

795c687

…vice

abhishekrb19 approved these changes Mar 4, 2026

View reviewed changes

Comment thread web-console/src/druid-models/web-console-config/web-console-config.tsx Outdated

aho135 added 3 commits March 4, 2026 15:28

Address review comment

d70eae8

formatting

2bb1c86

Remove special character

2542248

aho135 merged commit a131ad2 into apache:master Mar 5, 2026
37 checks passed

github-actions Bot added this to the 37.0.0 milestone Mar 5, 2026

Uh oh!

Conversation

aho135 commented Feb 28, 2026

Description

Release note

Key changed/added classes in this PR

Uh oh!

abhishek-chouhan commented Mar 2, 2026

Uh oh!

aho135 commented Mar 2, 2026

Uh oh!

abhishekrb19 Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

aho135 Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

aho135 Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

abhishekrb19 Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

aho135 Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

abhishekrb19 Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aho135 Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

abhishek-chouhan commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abhishekrb19 Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

aho135 Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

abhishekrb19 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

abhishekrb19 Mar 4, 2026 •

edited

Loading

abhishek-chouhan commented Mar 2, 2026 •

edited

Loading

aho135 Mar 4, 2026 •

edited

Loading