Skip to content

Conversation

@Jackie-Jiang
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang commented Feb 25, 2022

Description

Fix #7578

Currently, when a new broker joins the cluster, or the tags for a broker is changed, broker resource won't be updated automatically. Users need to either rebuild broker resource for each table in that broker tenant, or wait for the periodic task to fix the broker resource. Before that, the changes for the broker won't be reflected.

This PR makes the following enhancements:

  • Add a boolean flag updateBrokerResource to create/update instance rest APIs to automatically update the broker resource when enabled (disabled by default to keep the current behavior because updating broker resource can be costly for large cluster)
  • Add a rest API to update the broker resource for a specified broker
  • For single-tenant cluster, when broker joins the cluster for the first time as DefaultTenant, automatically update the broker resource so that it can build the routing tables properly.

Release Notes

Add an optional boolean query parameter updateBrokerResource to the following rest APIs:

  • POST /instances: Add an instance
  • PUT /instances/{instanceName}: Update an instance
  • PUT /instances/{instanceName}/updateTags: Update the tags for an instance

Add a rest API to update the broker resource for a specified broker:

  • POST /instances/{instanceName}/updateBrokerResource

For single-tenant cluster, when broker joins the cluster for the first time as DefaultTenant, automatically update the broker resource so that it can build the routing tables properly

@Jackie-Jiang Jackie-Jiang added the release-notes Referenced by PRs that need attention when compiling the next release notes label Feb 25, 2022
@codecov-commenter
Copy link

codecov-commenter commented Feb 25, 2022

Codecov Report

Merging #8249 (914b8ea) into master (27d690d) will increase coverage by 0.02%.
The diff coverage is 74.44%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #8249      +/-   ##
============================================
+ Coverage     70.74%   70.77%   +0.02%     
  Complexity     4238     4238              
============================================
  Files          1629     1629              
  Lines         85213    85364     +151     
  Branches      12830    12857      +27     
============================================
+ Hits          60287    60419     +132     
- Misses        20767    20774       +7     
- Partials       4159     4171      +12     
Flag Coverage Δ
integration1 28.87% <43.17%> (+0.03%) ⬆️
integration2 27.47% <23.34%> (-0.14%) ⬇️
unittests1 66.88% <7.22%> (-0.07%) ⬇️
unittests2 14.16% <47.57%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...er/api/resources/PinotInstanceRestletResource.java 58.41% <50.00%> (-5.79%) ⬇️
...g/apache/pinot/common/utils/helix/HelixHelper.java 51.42% <64.58%> (+6.28%) ⬆️
...ache/pinot/common/metadata/ZKMetadataProvider.java 79.86% <80.00%> (+0.72%) ⬆️
...ntroller/helix/core/PinotHelixResourceManager.java 67.01% <82.55%> (+0.95%) ⬆️
...e/pinot/broker/broker/helix/BaseBrokerStarter.java 78.17% <88.88%> (+1.95%) ⬆️
.../controller/helix/ControllerRequestURLBuilder.java 83.72% <100.00%> (+0.25%) ⬆️
...ller/helix/core/minion/TaskTypeMetricsUpdater.java 80.00% <0.00%> (-20.00%) ⬇️
...rg/apache/pinot/server/starter/ServerInstance.java 88.78% <0.00%> (-2.81%) ⬇️
...form/function/BinaryOperatorTransformFunction.java 41.77% <0.00%> (-2.67%) ⬇️
...core/query/pruner/SelectionQuerySegmentPruner.java 86.36% <0.00%> (-2.28%) ⬇️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27d690d...914b8ea. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes Referenced by PRs that need attention when compiling the next release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ensure broker rebuild is done during initialization

3 participants