Repository navigation
Commit 03f4c12
[fix](catalog) hudi H4: lowercase JNI reader column names
Batch-0 (apache#65185 reverify) H4. After the HMS SPI cutover, a hudi-on-HMS
table with mixed-case Avro columns (e.g. Id/Name/Addr) crashed every
MOR/JNI split.
Root cause: HudiScanPlanProvider.planScan built the JNI reader column
list with `.map(Schema.Field::name)` (original case). BE
HadoopHudiJniScanner.initRequiredColumnsAndTypes keys hudiColNameToType
by these names, then resolves each requiredField with an EXACT lower-case
containsKey; mixed-case "Id" misses lower-case "id" -> throw. The two
sibling name paths were already lower-cased (avroSchemaToColumns:905,
HudiSchemaUtils.buildField:137); legacy HudiScanNode:223 also emits
lower-case names. This one path regressed.
Fix: extract the inline transform into a package-private static
`jniColumnNames(Schema)` that lower-cases each field name
(toLowerCase(Locale.ROOT)); column ORDER preserved so the parallel
columnTypes list stays positionally aligned. Extraction makes the
transform offline-unit-testable (planScan needs a live metaClient).
Test: HudiSchemaParityTest.jniColumnNamesAreLowerCased asserts the
mixed-case Id/Name/Addr fixture lower-cases. MOR/JNI end-to-end read
with mixed-case columns is live-gated (real hudi cluster).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NoUy5UkBMkyE9pqSH84CR31 parent 17a9aaf commit 03f4c12
2 files changed
Lines changed: 35 additions & 2 deletions
File tree
- fe/fe-connector/fe-connector-hudi/src
- main/java/org/apache/doris/connector/hudi
- test/java/org/apache/doris/connector/hudi
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
| |||
177 | 178 | | |
178 | 179 | | |
179 | 180 | | |
180 | | - | |
181 | | - | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
182 | 187 | | |
183 | 188 | | |
184 | 189 | | |
| |||
827 | 832 | | |
828 | 833 | | |
829 | 834 | | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
830 | 851 | | |
831 | 852 | | |
832 | 853 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
151 | 163 | | |
152 | 164 | | |
153 | 165 | | |
| |||
0 commit comments