Skip to content

Commit 26eebd6

Browse files
morningmanclaude
andcommitted
[test](paimon) Cover chain table payloads in the backend manifest cap
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: The backend scanner bounds Paimon's manifest parallelism by rebuilding the transported table, and this PR made that rebuild keep a chain table's ChainGroupReadTable. Only the plain-table overload was covered. A system table over a chain table ($partitions, $files, ...) is rebuilt from the transported source instead, and the backend resolves $partitions itself, so losing the group there drops the partitions only the delta branch holds. A backend upgraded before the FE also still receives the payload of the previous FE, in which the chain group has been replaced by a plain table at the snapshot branch, and must keep reading it as before. ### Release note None ### Check List (For Author) - Test: Unit Test - PaimonJniScannerTest .testBackendCapKeepsTheChainGroupOfAChainTableSystemSource: a $partitions wrapper rebuilt from a chain source keeps the group. - PaimonJniScannerTest .testBackendCapStillReadsTheChainTablePayloadOfAnOlderFrontend: the previous FE's pair stays a plain fallback pair. - Behavior changed: No - Does this need documentation: No Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent f64111f commit 26eebd6

1 file changed

Lines changed: 36 additions & 0 deletions

File tree

‎fe/be-java-extensions/paimon-scanner/src/test/java/org/apache/doris/paimon/PaimonJniScannerTest.java‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,6 +334,42 @@ public void testBackendManifestCapKeepsTheChainGroupOfAChainTable(@TempDir Path
334334
.get(CoreOptions.SCAN_MANIFEST_PARALLELISM.key()));
335335
}
336336

337+
@Test
338+
public void testBackendCapKeepsTheChainGroupOfAChainTableSystemSource(@TempDir Path dir) throws Exception {
339+
// A system table over a chain table ($partitions, $files, ...) is rebuilt from the transported source, and
340+
// the rebuilt wrapper must still read through the chain group: the backend resolves $partitions, and
341+
// without the delta branch a partition only it holds goes missing. MUTATION: rebuilding the group as a
342+
// plain fallback pair -> red.
343+
FileStoreTable chain = new FallbackReadFileStoreTable(primaryKeyTable(dir, "main"),
344+
new ChainGroupReadTable(primaryKeyTable(dir, "snapshot"), primaryKeyTable(dir, "delta")), true);
345+
346+
Table safe = PaimonJniScanner.applyBackendManifestParallelism(
347+
SystemTableLoader.load("partitions", chain), "2", 4, chain, "partitions");
348+
349+
Field storeTable = safe.getClass().getDeclaredField("storeTable");
350+
storeTable.setAccessible(true);
351+
FileStoreTable group = ((FallbackReadFileStoreTable) storeTable.get(safe)).other();
352+
Assertions.assertTrue(group instanceof ChainGroupReadTable, group.getClass().getName());
353+
Assertions.assertEquals("2", ((ChainGroupReadTable) group).other().options()
354+
.get(CoreOptions.SCAN_MANIFEST_PARALLELISM.key()));
355+
}
356+
357+
@Test
358+
public void testBackendCapStillReadsTheChainTablePayloadOfAnOlderFrontend(@TempDir Path dir) {
359+
// An older FE sends a chain table with its group replaced by a plain table at the snapshot branch. A backend
360+
// upgraded before the FE keeps reading that payload as it did. MUTATION: rebuilding a pair as a chain group
361+
// whatever it holds -> red.
362+
FileStoreTable flattened = new FallbackReadFileStoreTable(primaryKeyTable(dir, "main"),
363+
primaryKeyTable(dir, "snapshot"), true);
364+
365+
Table safe = PaimonJniScanner.applyBackendManifestParallelism(flattened, "2", 4);
366+
367+
Assertions.assertEquals(FallbackReadFileStoreTable.class, safe.getClass());
368+
FileStoreTable other = ((FallbackReadFileStoreTable) safe).other();
369+
Assertions.assertFalse(other instanceof FallbackReadFileStoreTable, other.getClass().getName());
370+
Assertions.assertEquals("2", other.options().get(CoreOptions.SCAN_MANIFEST_PARALLELISM.key()));
371+
}
372+
337373
@Test
338374
public void testAdvertisedFeCapStillChecksSerializedChildren() {
339375
FileStoreTable main = serializableFileStoreTable(Collections.emptyMap());

0 commit comments

Comments
 (0)