Repository navigation
Commit cf79a1a
[fix](paimon) Check binary partitions of fallback splits under the options of their branch
### What problem does this PR solve?
Issue Number: None
Related PR: #69571
Problem Summary: Follow-up to the scan-planning check of binary partitions
in this PR, for two cases found in review.
The check takes the naming option and the file schemas from the branch
that produced each split, but only when the table has a schema
dictionary. For $audit_log and $row_tracking, which are read through JNI,
there is none, so the check used the wrapper's own options, and an
AuditLogTable reports the options of the fallback pair, which are the
main branch's. With scan.fallback-branch set, a partition missing on the
main branch is still planned and read under the fallback branch's
options. A table whose main branch keeps the default
'partition.legacy-name' = 'true' and holds only a NULL partition, while
its fallback branch uses 'false' and holds a non-NULL binary partition,
was therefore rejected for $audit_log although Paimon reads the
fallback rows; in the reverse layout the scan planned and Paimon's reader
failed with FileNotFoundException at execution. For the wrappers read
through JNI the check now resolves the table that produced the splits
from the wrapper's pinned source, as the timestamp-history checks already
do, and picks each split's branch from it as it does for data tables.
$binlog exposes its partition columns as arrays and is not checked.
A scan.file-creation-time-millis read plans a fallback table through the
pair's SnapshotReader, which reads the wrapped branch and returns plain
DataSplits. Picking the branch cast every split to FallbackSplit, so such
a scan of a table with a non-NULL binary partition failed with a
ClassCastException (the same cast on the native timestamp path predates
this PR). A split that is not a FallbackSplit now maps to the wrapped
branch.
### Release note
None (follow-up inside the PR)
### Check List (For Author)
- Test: Unit Test
- PaimonScanPlanProviderTest: a fallback pair whose main branch holds only a NULL partition and whose fallback branch holds a non-NULL binary partition plans the data table and $audit_log, and the $audit_log splits read back both rows through Paimon, when the fallback branch names partitions by bytes; both fail at planning with the legacy-name error when only the main branch does; a scan.file-creation-time-millis read of a fallback pair with a non-NULL binary partition plans the main branch's file
- fe-connector-paimon module tests: 853 run, 0 failures, 0 errors, 1 skipped (pre-existing)
- Behavior changed: Yes. $audit_log and $row_tracking of a fallback table check binary partitions under the options of the branch that produced each split, and a file-creation-time read of a fallback table with a binary partition no longer fails with a ClassCastException.
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>1 parent 65a1682 commit cf79a1a
2 files changed
Lines changed: 117 additions & 4 deletions
File tree
- fe/fe-connector/fe-connector-paimon/src
- main/java/org/apache/doris/connector/paimon
- test/java/org/apache/doris/connector/paimon
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
992 | 992 | | |
993 | 993 | | |
994 | 994 | | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
995 | 1001 | | |
996 | 1002 | | |
997 | 1003 | | |
| |||
1006 | 1012 | | |
1007 | 1013 | | |
1008 | 1014 | | |
1009 | | - | |
| 1015 | + | |
1010 | 1016 | | |
1011 | 1017 | | |
1012 | 1018 | | |
| |||
2294 | 2300 | | |
2295 | 2301 | | |
2296 | 2302 | | |
| 2303 | + | |
| 2304 | + | |
| 2305 | + | |
| 2306 | + | |
| 2307 | + | |
2297 | 2308 | | |
2298 | 2309 | | |
2299 | 2310 | | |
| |||
2316 | 2327 | | |
2317 | 2328 | | |
2318 | 2329 | | |
2319 | | - | |
2320 | | - | |
2321 | | - | |
| 2330 | + | |
| 2331 | + | |
| 2332 | + | |
| 2333 | + | |
2322 | 2334 | | |
2323 | 2335 | | |
2324 | 2336 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| 83 | + | |
83 | 84 | | |
84 | 85 | | |
85 | 86 | | |
| |||
1085 | 1086 | | |
1086 | 1087 | | |
1087 | 1088 | | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
| 1093 | + | |
| 1094 | + | |
| 1095 | + | |
| 1096 | + | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
| 1102 | + | |
| 1103 | + | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
| 1138 | + | |
| 1139 | + | |
| 1140 | + | |
| 1141 | + | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
| 1146 | + | |
| 1147 | + | |
| 1148 | + | |
| 1149 | + | |
| 1150 | + | |
| 1151 | + | |
| 1152 | + | |
| 1153 | + | |
| 1154 | + | |
| 1155 | + | |
| 1156 | + | |
| 1157 | + | |
| 1158 | + | |
| 1159 | + | |
| 1160 | + | |
| 1161 | + | |
| 1162 | + | |
| 1163 | + | |
| 1164 | + | |
| 1165 | + | |
| 1166 | + | |
| 1167 | + | |
| 1168 | + | |
| 1169 | + | |
| 1170 | + | |
| 1171 | + | |
| 1172 | + | |
| 1173 | + | |
1088 | 1174 | | |
1089 | 1175 | | |
1090 | 1176 | | |
| |||
1120 | 1206 | | |
1121 | 1207 | | |
1122 | 1208 | | |
| 1209 | + | |
| 1210 | + | |
| 1211 | + | |
| 1212 | + | |
| 1213 | + | |
| 1214 | + | |
| 1215 | + | |
| 1216 | + | |
| 1217 | + | |
| 1218 | + | |
| 1219 | + | |
| 1220 | + | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
1123 | 1224 | | |
1124 | 1225 | | |
1125 | 1226 | | |
| |||
0 commit comments