Repository navigation
Commit f1104a6
[fix](catalog) fe-connector-iceberg: reject case-insensitive column-name collisions on the top-level ALTER ops
Port gap from apache#65329. Upstream `IcebergMetadataOps` called
`validateNoCaseInsensitiveSiblingCollision` from five sites: flat ADD, nested ADD,
the ADD COLUMNS batch, flat RENAME and nested RENAME. The connector port carried
only the two NESTED sites into `IcebergNestedColumnEvolution`; the top-level ops in
`CatalogBackedIcebergCatalogOps` went straight to `UpdateSchema`.
Iceberg matches field names case-sensitively (`SchemaUpdate` sets `caseSensitive =
true`) while Doris column names are case-insensitive, so nothing caught the
mismatch: against a Spark-created table with a field `Id`, `ADD COLUMN id` silently
committed a second column, and `RENAME COLUMN label ...` died with iceberg's raw
"Cannot rename missing column: label" because the real field is `Label`.
Wire the three top-level ops to the validators the nested arm already uses:
- addColumn - reject a name colliding case-insensitively with an existing field.
- addColumns - same per column, plus reject two names of one request that collide
with each other; the whole batch is validated before anything is
staged, so a partly-valid request commits nothing.
- renameColumn - resolve the SOURCE name case-insensitively (a pure re-casing
rename is let through by the field-identity escape) and restate
identifier-field paths via the existing `applyRenameColumn`.
`validateNoCaseInsensitiveSiblingCollision` becomes package-private; its
`parentPath.isEmpty()` top-level arm was already written but unreachable.
Fixes external_table_p0/iceberg/test_iceberg_nested_schema_evolution_spark_doris_interop
line 273 (TeamCity Doris_External_Regression #1005291), and the four assertions
after it that the suite never reached.
Tests: 7 cases in CatalogBackedIcebergCatalogOpsColumnEvolutionTest against a real
InMemoryCatalog with a mixed-case fixture (Id / Label / Info.Metric); they fail on
the pre-fix code with the exact CI symptoms. Module suite 1145/1145 green
(cache-free run), full-reactor test-compile green.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VK7NGsHDmPGvpHFFK5ojej1 parent 7ff51a1 commit f1104a6
3 files changed
Lines changed: 164 additions & 6 deletions
File tree
- fe/fe-connector/fe-connector-iceberg/src
- main/java/org/apache/doris/connector/iceberg
- test/java/org/apache/doris/connector/iceberg
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
434 | 434 | | |
435 | 435 | | |
436 | 436 | | |
437 | | - | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
438 | 441 | | |
439 | 442 | | |
440 | 443 | | |
| |||
443 | 446 | | |
444 | 447 | | |
445 | 448 | | |
446 | | - | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
447 | 452 | | |
448 | 453 | | |
449 | 454 | | |
| |||
460 | 465 | | |
461 | 466 | | |
462 | 467 | | |
463 | | - | |
464 | | - | |
465 | | - | |
| 468 | + | |
466 | 469 | | |
467 | 470 | | |
468 | 471 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
52 | 57 | | |
53 | 58 | | |
54 | 59 | | |
| |||
277 | 282 | | |
278 | 283 | | |
279 | 284 | | |
280 | | - | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
281 | 324 | | |
282 | 325 | | |
283 | 326 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
134 | 246 | | |
135 | 247 | | |
136 | 248 | | |
| |||
0 commit comments