Repository navigation
Commit d0d4786
authored
Treat NaN floats at the same position as equal in
* Treat NaN floats at the same position as equal in `Record.__eq__`
`Record.__eq__` compared the underlying lists, so two separately decoded
NaN values compared unequal while their `str`-based hashes matched.
`DeleteFileIndex` keys partition-scoped position deletes by partition
`Record`, so those deletes never applied to data files in a NaN identity
partition, and scans returned deleted rows. Commit validation missed
concurrently deleted files in NaN partitions for the same reason.
Compare elementwise when the lists differ, counting NaN floats at the
same position as equal. The hash is unchanged.
Closes apache#4279
* Compare nested records once in `Record.__eq__`
Drop the list fast path so nested records are not compared twice when they differ.
* Simplify `Record.__eq__` with a `_same_value` helper
Move the per-position comparison into a helper so `__eq__` reads as a single `all()` over the fields.Record.__eq__ (apache#4280)1 parent 40ebd54 commit d0d4786
4 files changed
Lines changed: 85 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
173 | 174 | | |
174 | 175 | | |
175 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
176 | 185 | | |
177 | 186 | | |
178 | 187 | | |
| |||
196 | 205 | | |
197 | 206 | | |
198 | 207 | | |
199 | | - | |
200 | | - | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
201 | 212 | | |
202 | 213 | | |
203 | 214 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| 39 | + | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| |||
235 | 237 | | |
236 | 238 | | |
237 | 239 | | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
238 | 271 | | |
239 | 272 | | |
240 | 273 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
0 commit comments