Repository navigation
Commit 897bb25
authored
[fix](iceberg) Validate Parquet compression before file creation (apache#69376)
### What problem does this PR solve?
Issue Number: N/A
Related PR: apache#68854, apache#69338
Problem Summary:
The Iceberg data, position-delete, and Hive Parquet writers created
their output file before validating the requested Parquet compression
codec. For an unsupported codec, the Iceberg data writer then attempted
to delete an open file, while the other writers returned after creating
the file. This made error cleanup depend on filesystem behavior and
could leave an orphan file. Resolve the codec before calling
`create_file` so invalid codecs fail without creating any output file,
and add a regression test for the failure path.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test: added `UnsupportedParquetCodecDoesNotCreateFile`; CI
will run it.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label1 parent dac77e0 commit 897bb25
4 files changed
Lines changed: 59 additions & 12 deletions
File tree
- be
- src/exec/sink/writer
- iceberg
- test/exec/sink/writer/iceberg
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
84 | 83 | | |
85 | 84 | | |
86 | 85 | | |
87 | 86 | | |
88 | 87 | | |
89 | 88 | | |
| 89 | + | |
| 90 | + | |
90 | 91 | | |
91 | 92 | | |
92 | 93 | | |
| |||
96 | 97 | | |
97 | 98 | | |
98 | 99 | | |
| 100 | + | |
| 101 | + | |
99 | 102 | | |
100 | 103 | | |
101 | 104 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
100 | | - | |
101 | 100 | | |
102 | 101 | | |
103 | 102 | | |
104 | 103 | | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
113 | 108 | | |
114 | 109 | | |
115 | | - | |
| 110 | + | |
116 | 111 | | |
117 | 112 | | |
118 | 113 | | |
| |||
125 | 120 | | |
126 | 121 | | |
127 | 122 | | |
| 123 | + | |
| 124 | + | |
128 | 125 | | |
129 | 126 | | |
130 | 127 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
88 | | - | |
89 | 88 | | |
90 | 89 | | |
91 | 90 | | |
92 | 91 | | |
93 | 92 | | |
| 93 | + | |
| 94 | + | |
94 | 95 | | |
95 | 96 | | |
96 | 97 | | |
| |||
107 | 108 | | |
108 | 109 | | |
109 | 110 | | |
| 111 | + | |
| 112 | + | |
110 | 113 | | |
111 | 114 | | |
112 | 115 | | |
113 | 116 | | |
114 | 117 | | |
115 | 118 | | |
| 119 | + | |
| 120 | + | |
116 | 121 | | |
117 | 122 | | |
118 | 123 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
205 | 206 | | |
206 | 207 | | |
207 | 208 | | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
208 | 250 | | |
209 | 251 | | |
210 | 252 | | |
| |||
0 commit comments