Repository navigation
Commit 99a6af8
authored
fix: reject NULL partition keys during partitioned writes (apache#24951)
## Which issue does this PR close?
Related to apache#18083. Full support for NULL partition values remains
covered by that issue.
## Rationale for this change
Partitioned writes read partition values without checking their
validity. A NULL integer partition key can be written under `p=0`, so
reading the output turns NULL into zero and silently changes the data.
For example:
```sql
CREATE TABLE src(p INT, v INT) AS VALUES (NULL, 1), (0, 2);
COPY src TO '/tmp/null_partition/' STORED AS PARQUET PARTITIONED BY (p);
```
Both rows can end up in the zero partition. This PR rejects the write
with `NULL values are not supported for partition column 'p'`.
## What changes are included in this PR?
- Check logical nulls before extracting partition keys, including NULL
dictionary values referenced by valid dictionary indices.
- Continue accepting nullable columns when the selected rows contain no
NULL values.
- Add focused unit tests and a `copy.slt` regression covering rejection
and a filtered successful write.
## What is the testing strategy for this PR?
- The rejection regressions fail without the production fix and pass
with it.
- Focused partition-key tests: 5 passed with `RUST_BACKTRACE=1`.
- `cargo clippy --all-targets --all-features -- -D warnings` passed.
- The extended workspace Rust suites passed after excluding four host
memory-measurement runners that also fail on current main in this
environment.
- The SQL logic harness passed all 513 files, including encrypted
Parquet coverage.
- `cargo fmt --all -- --check` and `git diff --check` passed.
## Are there any user-facing changes?
Partitioned writes containing NULL partition keys now return an
execution error. Callers can filter or replace these values before
writing. Validation happens per batch, so files written by earlier
batches can remain after an error.1 parent 17c6d74 commit 99a6af8
2 files changed
Lines changed: 112 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
42 | 44 | | |
43 | 45 | | |
44 | 46 | | |
| |||
392 | 394 | | |
393 | 395 | | |
394 | 396 | | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
395 | 402 | | |
396 | 403 | | |
397 | 404 | | |
| |||
595 | 602 | | |
596 | 603 | | |
597 | 604 | | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
45 | 67 | | |
46 | 68 | | |
47 | 69 | | |
| |||
0 commit comments