Disable reingestion for Pauseless dedup #15383
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #15383 +/- ##
============================================
+ Coverage 61.75% 63.21% +1.46%
- Complexity 207 1375 +1168
============================================
Files 2436 2812 +376
Lines 133233 158982 +25749
Branches 20636 24354 +3718
============================================
+ Hits 82274 100497 +18223
- Misses 44911 50894 +5983
- Partials 6048 7591 +1543
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
|
||
| if (isPauselessConsumptionEnabled) { | ||
| _llcRealtimeSegmentManager.repairSegmentsInErrorStateForPauselessConsumption(tableConfig.getTableName()); | ||
| if (!(isPartialUpsertEnabled || isDedupEnabled)) { |
There was a problem hiding this comment.
I think we might disable for enforceConsumptionInOrder = true case as well.
| return; | ||
| } | ||
|
|
||
| boolean isPartialUpsertEnabled = |
There was a problem hiding this comment.
We want to move this after the warning log below. We should also log something because we are not able to fix the segments. Probably an error log
There was a problem hiding this comment.
Probably also emit a metric for error segments not able to fix. We need to get immediate alert on it
5 participants
We cannot support reingestion with dedup since deduping requires in-order consumption.