Repository navigation
Commit 132905e
committed
Address review feedback on RowsGroupColumn PR
Fixes from @Rich-T-kid and @copilot-pull-request-reviewer:
* row.rs: correct the doc on `encode_array_if_necessary` — arrow's
`RowConverter` flattens dictionary / REE values at `append()` (during
row encoding), not "on the way out".
* row.rs: extend the recursion in `encode_array_if_necessary` to cover
`FixedSizeList`, `LargeList`, and `Map` in addition to `Struct` and
`List`. Without this, a nested key like `List<Dict<Int32, Utf8>>`
emits as `List<Utf8>` because the fallback arm just clones the
RowConverter-flattened array, giving a data-type mismatch against the
declared output type.
* row.rs: rename `dictionary_encode_if_necessary` → `encode_array_if_necessary`
since the helper also handles run-end encoding (and now the nested
container walks around them).
* row_backed.rs: add `build_preserves_list_of_dictionary_schema` — a
round-trip regression test that appends a `List<Dict<Int32, Utf8>>`
row via `RowsGroupColumn` and asserts the built array's data type
matches the declared schema (would fail with the pre-fix helper).
* multi_group_by/mod.rs: expand the doc on `canonical_grouping` to
spell out that the `GroupValues` trait deliberately does not fix the
order in which fresh group-ids are assigned — canonicalizing before
comparison is what lets the vectorized column path and the sequential
rows fallback be asserted equivalent.
* row_backed.rs (module doc): reword "instead of dragging the entire
key onto GroupValuesRows" → "instead of dragging both columns onto
GroupValuesRows" per Rich-T-kid's suggestion diff.1 parent cef4ed4 commit 132905e
3 files changed
Lines changed: 115 additions & 17 deletions
File tree
- datafusion/physical-plan/src/aggregates/group_values
- multi_group_by
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1398 | 1398 | | |
1399 | 1399 | | |
1400 | 1400 | | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
| 1407 | + | |
1401 | 1408 | | |
1402 | 1409 | | |
1403 | 1410 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
36 | | - | |
| 35 | + | |
| 36 | + | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
| 50 | + | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
| 121 | + | |
122 | 122 | | |
123 | 123 | | |
124 | 124 | | |
| |||
370 | 370 | | |
371 | 371 | | |
372 | 372 | | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
373 | 420 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | | - | |
| 20 | + | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
247 | 247 | | |
248 | 248 | | |
249 | 249 | | |
250 | | - | |
| 250 | + | |
251 | 251 | | |
252 | 252 | | |
253 | 253 | | |
| |||
269 | 269 | | |
270 | 270 | | |
271 | 271 | | |
272 | | - | |
273 | | - | |
274 | | - | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
275 | 276 | | |
276 | 277 | | |
277 | 278 | | |
278 | | - | |
| 279 | + | |
| 280 | + | |
279 | 281 | | |
280 | 282 | | |
281 | 283 | | |
| |||
286 | 288 | | |
287 | 289 | | |
288 | 290 | | |
289 | | - | |
| 291 | + | |
290 | 292 | | |
291 | 293 | | |
292 | 294 | | |
| |||
302 | 304 | | |
303 | 305 | | |
304 | 306 | | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
| 307 | + | |
309 | 308 | | |
310 | 309 | | |
311 | 310 | | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
312 | 356 | | |
313 | 357 | | |
314 | 358 | | |
| |||
320 | 364 | | |
321 | 365 | | |
322 | 366 | | |
323 | | - | |
| 367 | + | |
324 | 368 | | |
325 | 369 | | |
326 | 370 | | |
| |||
0 commit comments