Repository navigation
Commit 36bac83
[fix](iceberg) Convert DATE defaults before rebuilding a promoted equality-delete key
### What problem does this PR solve?
Issue Number: None
Related PR: #69536
Problem Summary: The previous commit lets a dropped Iceberg equality-delete
key take, from any retained schema, the v3 promotion of DATE to an unzoned
TIMESTAMP or TIMESTAMP_NS, and rebuilds the key's field with
Types.NestedField.from(field).ofType(type). The builder copies the field's
initial and write defaults unchanged, and NestedField's constructor casts
both to the new type with Literal.to. In Iceberg 1.12.0 a DATE literal
converts only to DATE, so the rebuild threw "Cannot cast default value to
timestamp". This happened on a v3 table whose DATE key has a default, was
promoted without a snapshot and was dropped before its equality delete
committed: the lineage selected the DATE definition, the retained
promotion widened it, and planning failed for every scan the delete
applies to. The SDK promotions (INT to LONG, FLOAT to DOUBLE, decimal
precision) convert their literals already, and a change of optionality
alone keeps the type, so only DATE defaults were affected.
The rebuild now converts a DATE default to midnight of its day in the
promoted TIMESTAMP or TIMESTAMP_NS before building the field. A date
outside the promoted type's range still fails, as the format requires for
this promotion.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- IcebergScanPlanProviderTest: a v3 DATE key with initial and write defaults, promoted to TIMESTAMP or TIMESTAMP_NS without a snapshot and dropped before its delete commits, keeps both defaults as midnight in the promoted type, and the scan dictionary carries the converted initial default
- fe-connector-iceberg module tests
- Behavior changed: No (fixes a regression of this PR's previous commit)
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>1 parent cbf21fb commit 36bac83
2 files changed
Lines changed: 73 additions & 2 deletions
File tree
- fe/fe-connector/fe-connector-iceberg/src
- main/java/org/apache/doris/connector/iceberg
- test/java/org/apache/doris/connector/iceberg
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| 82 | + | |
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| |||
92 | 93 | | |
93 | 94 | | |
94 | 95 | | |
| 96 | + | |
95 | 97 | | |
96 | 98 | | |
97 | 99 | | |
| |||
2346 | 2348 | | |
2347 | 2349 | | |
2348 | 2350 | | |
2349 | | - | |
| 2351 | + | |
| 2352 | + | |
2350 | 2353 | | |
2351 | 2354 | | |
2352 | 2355 | | |
| |||
2368 | 2371 | | |
2369 | 2372 | | |
2370 | 2373 | | |
2371 | | - | |
| 2374 | + | |
| 2375 | + | |
| 2376 | + | |
| 2377 | + | |
2372 | 2378 | | |
2373 | 2379 | | |
2374 | 2380 | | |
| |||
2388 | 2394 | | |
2389 | 2395 | | |
2390 | 2396 | | |
| 2397 | + | |
| 2398 | + | |
| 2399 | + | |
| 2400 | + | |
| 2401 | + | |
| 2402 | + | |
| 2403 | + | |
| 2404 | + | |
| 2405 | + | |
| 2406 | + | |
| 2407 | + | |
| 2408 | + | |
| 2409 | + | |
| 2410 | + | |
2391 | 2411 | | |
2392 | 2412 | | |
2393 | 2413 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
| 92 | + | |
92 | 93 | | |
93 | 94 | | |
94 | 95 | | |
| |||
2193 | 2194 | | |
2194 | 2195 | | |
2195 | 2196 | | |
| 2197 | + | |
| 2198 | + | |
| 2199 | + | |
| 2200 | + | |
| 2201 | + | |
| 2202 | + | |
| 2203 | + | |
| 2204 | + | |
| 2205 | + | |
| 2206 | + | |
| 2207 | + | |
| 2208 | + | |
| 2209 | + | |
| 2210 | + | |
| 2211 | + | |
| 2212 | + | |
| 2213 | + | |
| 2214 | + | |
| 2215 | + | |
| 2216 | + | |
| 2217 | + | |
| 2218 | + | |
| 2219 | + | |
| 2220 | + | |
| 2221 | + | |
| 2222 | + | |
| 2223 | + | |
| 2224 | + | |
| 2225 | + | |
| 2226 | + | |
| 2227 | + | |
| 2228 | + | |
| 2229 | + | |
| 2230 | + | |
| 2231 | + | |
| 2232 | + | |
| 2233 | + | |
| 2234 | + | |
| 2235 | + | |
| 2236 | + | |
| 2237 | + | |
| 2238 | + | |
| 2239 | + | |
| 2240 | + | |
| 2241 | + | |
| 2242 | + | |
| 2243 | + | |
| 2244 | + | |
| 2245 | + | |
| 2246 | + | |
2196 | 2247 | | |
2197 | 2248 | | |
2198 | 2249 | | |
| |||
0 commit comments