Repository navigation
Commit b86f1f2
authored
fix: preserve functional dependencies across unnest (apache#25892)
## Which issue does this PR close?
- Closes apache#25844
## Rationale for this change
A list UNNEST can produce several rows for one input key. Retaining the
input's functional dependencies makes a later DISTINCT or GROUP BY
discard unnested values, and can make an ordered window use the wrong
frame. These queries return incorrect results.
## What changes are included in this PR?
- Remap inherited dependency indices to output positions after struct
expansion.
- Downgrade dependencies after list unnest, remove unnested list outputs
from dependency targets, and discard dependencies whose determinants are
unnested.
- Add SQL and plan-level regressions for distinct, grouping, window
frames, and shifted indices.
## What is the testing strategy for this PR?
- cargo fmt --all --check
- cargo test -p datafusion-expr --lib
unnest_remaps_functional_dependencies
- cargo test --profile ci --test sqllogictests --
functional_dependencies.slt
## Are there any user-facing changes?
Yes. A list unnest followed by DISTINCT, GROUP BY, or an ordered window
now returns the expected result.1 parent fbf7737 commit b86f1f2
2 files changed
Lines changed: 203 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5261 | 5261 | | |
5262 | 5262 | | |
5263 | 5263 | | |
5264 | | - | |
5265 | | - | |
| 5264 | + | |
| 5265 | + | |
| 5266 | + | |
| 5267 | + | |
| 5268 | + | |
| 5269 | + | |
| 5270 | + | |
| 5271 | + | |
| 5272 | + | |
| 5273 | + | |
| 5274 | + | |
| 5275 | + | |
| 5276 | + | |
| 5277 | + | |
| 5278 | + | |
| 5279 | + | |
| 5280 | + | |
| 5281 | + | |
| 5282 | + | |
| 5283 | + | |
| 5284 | + | |
| 5285 | + | |
| 5286 | + | |
| 5287 | + | |
| 5288 | + | |
| 5289 | + | |
| 5290 | + | |
| 5291 | + | |
| 5292 | + | |
| 5293 | + | |
| 5294 | + | |
| 5295 | + | |
| 5296 | + | |
| 5297 | + | |
| 5298 | + | |
| 5299 | + | |
| 5300 | + | |
| 5301 | + | |
| 5302 | + | |
| 5303 | + | |
| 5304 | + | |
5266 | 5305 | | |
5267 | 5306 | | |
5268 | 5307 | | |
| |||
5368 | 5407 | | |
5369 | 5408 | | |
5370 | 5409 | | |
5371 | | - | |
| 5410 | + | |
5372 | 5411 | | |
5373 | 5412 | | |
5374 | 5413 | | |
| |||
5448 | 5487 | | |
5449 | 5488 | | |
5450 | 5489 | | |
| 5490 | + | |
| 5491 | + | |
| 5492 | + | |
| 5493 | + | |
| 5494 | + | |
| 5495 | + | |
| 5496 | + | |
| 5497 | + | |
| 5498 | + | |
| 5499 | + | |
| 5500 | + | |
| 5501 | + | |
| 5502 | + | |
| 5503 | + | |
| 5504 | + | |
| 5505 | + | |
| 5506 | + | |
| 5507 | + | |
| 5508 | + | |
| 5509 | + | |
| 5510 | + | |
| 5511 | + | |
| 5512 | + | |
| 5513 | + | |
| 5514 | + | |
| 5515 | + | |
| 5516 | + | |
| 5517 | + | |
| 5518 | + | |
| 5519 | + | |
| 5520 | + | |
| 5521 | + | |
| 5522 | + | |
| 5523 | + | |
| 5524 | + | |
| 5525 | + | |
| 5526 | + | |
| 5527 | + | |
| 5528 | + | |
| 5529 | + | |
| 5530 | + | |
| 5531 | + | |
| 5532 | + | |
| 5533 | + | |
| 5534 | + | |
| 5535 | + | |
| 5536 | + | |
| 5537 | + | |
| 5538 | + | |
| 5539 | + | |
| 5540 | + | |
| 5541 | + | |
| 5542 | + | |
| 5543 | + | |
| 5544 | + | |
| 5545 | + | |
| 5546 | + | |
| 5547 | + | |
| 5548 | + | |
| 5549 | + | |
| 5550 | + | |
| 5551 | + | |
| 5552 | + | |
| 5553 | + | |
| 5554 | + | |
| 5555 | + | |
| 5556 | + | |
| 5557 | + | |
| 5558 | + | |
| 5559 | + | |
| 5560 | + | |
| 5561 | + | |
| 5562 | + | |
| 5563 | + | |
5451 | 5564 | | |
5452 | 5565 | | |
5453 | 5566 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
466 | 466 | | |
467 | 467 | | |
468 | 468 | | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
469 | 556 | | |
470 | 557 | | |
471 | 558 | | |
| |||
0 commit comments