Skip to content

Commit 1145d3d

Browse files
committed
[fix](fe) Recheck split fetch errors after awaited range
### What problem does this PR solve? Issue Number: None Problem Summary: A one-split BE fetch can wait for a range, then receive that range after its producer has already published a failure or cancellation. The new per-split queue path appended the awaited range and returned as soon as the request filled, without rechecking terminal state. Recheck after notifying queue space so an already-failed assignment returns its error and a stopped assignment discards the range. A deterministic two-case test reproduced the previous incorrect return and passes with the fix. ### Release note None ### Check List (For Author) - Test: Unit Test. The new race test failed in both cases before the fix, then passed in both cases afterward. All 33 SplitSourceTest cases passed using the prior Surefire classpath and JUnit launcher. The local FE-core Maven route is blocked by unresolved snapshot reactor artifacts; the full reactor needs an x86 gRPC generator unavailable on this ARM host. - Behavior changed: Yes (an already-failed or cancelled one-split fetch no longer returns OK with a range) - Does this need documentation: No
1 parent 5a8470c commit 1145d3d

2 files changed

Lines changed: 43 additions & 0 deletions

File tree

‎fe/fe-core/src/main/java/org/apache/doris/datasource/split/SplitSource.java‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,13 @@ public synchronized List<TScanRangeLocations> getNextBatch(int maxBatchSize) thr
124124
TScanRangeLocations split = splitAssignment.awaitSplit(backend, remainingNanos);
125125
if (split != null) {
126126
splitAssignment.notifyQueueSpace(backend);
127+
// The awakened producer may fail or stop before this one-split fetch returns.
128+
splitAssignment.getAssignedSplits(backend);
129+
if (splitAssignment.isStop()) {
130+
scanRanges.clear();
131+
isLastBatch.set(true);
132+
break;
133+
}
127134
scanRanges.add(split);
128135
}
129136
} catch (InterruptedException e) {

‎fe/fe-core/src/test/java/org/apache/doris/datasource/split/SplitSourceTest.java‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,42 @@ void bulkDrainObservesTerminalStatePublishedByCapacityCallback(boolean fail) thr
248248
}
249249
}
250250

251+
@ParameterizedTest
252+
@ValueSource(booleans = {false, true})
253+
void awaitedSingleSplitObservesTerminalStateBeforeReturning(boolean fail) throws Exception {
254+
assignment = Mockito.spy(assignment);
255+
CountDownLatch waiting = new CountDownLatch(1);
256+
Mockito.doAnswer(invocation -> {
257+
waiting.countDown();
258+
TScanRangeLocations split = (TScanRangeLocations) invocation.callRealMethod();
259+
if (fail) {
260+
assignment.setException(new UserException("awaited split failure"));
261+
} else {
262+
assignment.stop();
263+
}
264+
return split;
265+
}).when(assignment).awaitSplit(Mockito.eq(backend), Mockito.anyLong());
266+
SplitSource source = new SplitSource(backend, assignment, 1000, 1);
267+
ExecutorService executor = Executors.newSingleThreadExecutor();
268+
try {
269+
Future<List<TScanRangeLocations>> fetch = executor.submit(() -> source.getNextBatch(1));
270+
Assertions.assertTrue(waiting.await(5, TimeUnit.SECONDS));
271+
Assertions.assertEquals(1, assignment.offerBatch(backend,
272+
Collections.singletonList(ranges.get(0)), 0, () -> {}));
273+
if (fail) {
274+
ExecutionException error = Assertions.assertThrows(ExecutionException.class,
275+
() -> fetch.get(5, TimeUnit.SECONDS));
276+
Assertions.assertInstanceOf(UserException.class, error.getCause());
277+
Assertions.assertTrue(error.getCause().getMessage().contains("awaited split failure"));
278+
} else {
279+
Assertions.assertTrue(fetch.get(5, TimeUnit.SECONDS).isEmpty());
280+
Assertions.assertTrue(source.getNextBatch(1).isEmpty());
281+
}
282+
} finally {
283+
executor.shutdownNow();
284+
}
285+
}
286+
251287
@Test
252288
void initialFetchLimitIsIndependentOfProductionAndSubsequentFetches() throws Exception {
253289
// All eight splits are already produced without a BE request, including a partially drained batch.

0 commit comments

Comments
 (0)