update target tier for segments if tierConfigs is provided #10642
Conversation
Codecov Report
@@ Coverage Diff @@
## master #10642 +/- ##
=========================================
Coverage 70.31% 70.32%
Complexity 6495 6495
=========================================
Files 2106 2106
Lines 113381 113374 -7
Branches 17090 17089 -1
=========================================
- Hits 79728 79725 -3
+ Misses 28059 28052 -7
- Partials 5594 5597 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 26 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
| relocate = true; | ||
| LOGGER.info("Relocating COMPLETED segments for table: {}", tableNameWithType); | ||
| } | ||
| if (_enableLocalTierMigration) { |
There was a problem hiding this comment.
multi volume feature was being controlled by this flag?
There was a problem hiding this comment.
this flag controls the controller side logic - checking segment current vs. target tier and issue segment reload msg.
There was a problem hiding this comment.
on the server side, this TierBasedSegmentDirectoryLoader should be used, instead of the default loader. The former one sets the correct tier on the segment for the controller side logic to check.
This PR tries to make index config tier overwriting #10553 more robust. Turns out the fix in #10642 is not enough, e.g. if one updates tierConfigs in table config and calls rebalance API to move segments around and it happens before SegmentRelocator task kicks in to update the segment target tiers, it's still possible that segments are loaded on the new servers without using the tier specific index configs. So moving the logic that updates segment target tier into rebalanceTable method to be shared by both table rebalance API and SegmentRelocator periodic task.
3 participants
SegmentRelocator should check and update segment target tier when tierConfigs is provided, so that segment gets its right target tier set in its ZK metadata to make features like index config tier overwrites (#10553) work properly. Previously, target tier was only updated for the multi-volume feature.