Repository navigation
Commit fab1e0c
committed
[fix](variant) Preserve inferred values across writer chunks
### What problem does this PR solve?
Related PR: apache#69221
Problem Summary: Extracted VARIANT chunks inherit a type without validating it against earlier values. Decimal widening can therefore filter valid rows as cast NULLs, and nullable-array type merging can reduce element scale. Resolve inferred promotions against retained chunks before committing any cast, preserving whole-path JSONB fallback and the active chunk's original Decimal representation. Unwrap nullable types before array/Decimal dispatch. Keep explicit typed-storage cast-null filtering unchanged.
### Release note
Preserve VARIANT Decimal values and decimal-array precision during chunked writes.
### Check List (For Author)
- Test: ASAN BE unit tests: three new tests fail before the fix and pass afterward; related writer, path-builder, shredder and compatibility tests: 103 passed, 2 skipped because NestedGroup write support is unavailable. Local UT uses GLIBC_COMPATIBILITY=OFF to avoid duplicate toolchain compatibility symbols. clang-format 16, build hygiene and clang-tidy on changed lines passed.
- Behavior changed: Yes, inferred cross-chunk conversions preserve valid values and Decimal array scale
- Does this need documentation: No1 parent cf1db49 commit fab1e0c
4 files changed
Lines changed: 98 additions & 23 deletions
File tree
- be/src/storage/segment
- variant
- v2
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
659 | 659 | | |
660 | 660 | | |
661 | 661 | | |
| 662 | + | |
662 | 663 | | |
663 | 664 | | |
664 | 665 | | |
| |||
678 | 679 | | |
679 | 680 | | |
680 | 681 | | |
681 | | - | |
682 | | - | |
| 682 | + | |
| 683 | + | |
683 | 684 | | |
684 | 685 | | |
685 | 686 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
371 | 371 | | |
372 | 372 | | |
373 | 373 | | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
374 | 379 | | |
375 | 380 | | |
376 | 381 | | |
| |||
1082 | 1087 | | |
1083 | 1088 | | |
1084 | 1089 | | |
1085 | | - | |
1086 | | - | |
1087 | | - | |
1088 | | - | |
1089 | | - | |
1090 | | - | |
| 1090 | + | |
1091 | 1091 | | |
1092 | 1092 | | |
1093 | 1093 | | |
1094 | | - | |
| 1094 | + | |
1095 | 1095 | | |
1096 | 1096 | | |
1097 | 1097 | | |
1098 | 1098 | | |
1099 | 1099 | | |
1100 | 1100 | | |
1101 | | - | |
| 1101 | + | |
1102 | 1102 | | |
1103 | 1103 | | |
1104 | 1104 | | |
1105 | | - | |
| 1105 | + | |
1106 | 1106 | | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
1107 | 1118 | | |
1108 | 1119 | | |
1109 | 1120 | | |
1110 | 1121 | | |
1111 | 1122 | | |
1112 | 1123 | | |
1113 | | - | |
1114 | | - | |
1115 | | - | |
| 1124 | + | |
1116 | 1125 | | |
1117 | 1126 | | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
1118 | 1138 | | |
1119 | 1139 | | |
1120 | 1140 | | |
| |||
1164 | 1184 | | |
1165 | 1185 | | |
1166 | 1186 | | |
1167 | | - | |
| 1187 | + | |
| 1188 | + | |
1168 | 1189 | | |
1169 | 1190 | | |
1170 | 1191 | | |
| |||
1193 | 1214 | | |
1194 | 1215 | | |
1195 | 1216 | | |
1196 | | - | |
| 1217 | + | |
1197 | 1218 | | |
1198 | 1219 | | |
1199 | 1220 | | |
1200 | 1221 | | |
1201 | 1222 | | |
1202 | 1223 | | |
1203 | | - | |
| 1224 | + | |
1204 | 1225 | | |
1205 | 1226 | | |
1206 | 1227 | | |
| |||
1211 | 1232 | | |
1212 | 1233 | | |
1213 | 1234 | | |
1214 | | - | |
| 1235 | + | |
1215 | 1236 | | |
1216 | 1237 | | |
1217 | 1238 | | |
| |||
1236 | 1257 | | |
1237 | 1258 | | |
1238 | 1259 | | |
1239 | | - | |
| 1260 | + | |
1240 | 1261 | | |
1241 | 1262 | | |
1242 | 1263 | | |
1243 | | - | |
| 1264 | + | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
1244 | 1279 | | |
1245 | 1280 | | |
1246 | 1281 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
57 | 62 | | |
58 | 63 | | |
59 | 64 | | |
| |||
62 | 67 | | |
63 | 68 | | |
64 | 69 | | |
65 | | - | |
| 70 | + | |
66 | 71 | | |
67 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
68 | 77 | | |
69 | 78 | | |
70 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
612 | 612 | | |
613 | 613 | | |
614 | 614 | | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
615 | 620 | | |
616 | 621 | | |
617 | 622 | | |
| |||
621 | 626 | | |
622 | 627 | | |
623 | 628 | | |
624 | | - | |
| 629 | + | |
| 630 | + | |
625 | 631 | | |
626 | 632 | | |
627 | 633 | | |
| |||
657 | 663 | | |
658 | 664 | | |
659 | 665 | | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
660 | 690 | | |
661 | 691 | | |
662 | 692 | | |
| |||
0 commit comments