Repository navigation
Commit 6185dee
fix: return an error for an invalid CSV null_regex instead of panicking (apache#25261)
## Which issue does this PR close?
- Closes apache#25260.
## Rationale for this change
A malformed `null_regex` panics the query task instead of returning an
error.
`CsvFormat::infer_schema_from_stream` compiled the pattern with
```rust
let regex = Regex::new(null_regex.as_str())
.expect("Unable to parse CSV null regex.");
```
so any pattern the `regex` crate rejects aborted the task. It is
reachable
straight from SQL, through a `CREATE EXTERNAL TABLE` that leaves its
columns to
schema inference:
```sql
CREATE EXTERNAL TABLE t STORED AS CSV LOCATION 'data.csv'
OPTIONS ('format.has_header' 'true', 'format.null_regex' '(');
```
```
task 9 panicked with message "Unable to parse CSV null regex.: Syntax(
regex parse error:
(
^
error: unclosed group
)"
```
An invalid regex is a bad option value, not an internal invariant, so it
should
come back as an error naming the pattern.
## What changes are included in this PR?
The regex is compiled once, before the per-chunk loop, and a failure is
propagated with `exec_datafusion_err!` rather than panicking. Hoisting
it also
stops the pattern being recompiled for every chunk of the inference
stream.
The error text matches the one used on the read side in apache#25254, so the
same bad
option reads the same whichever path hits it first.
## What is the testing strategy for this PR?
A `statement error Unable to parse CSV null regex` case in
`datafusion/sqllogictest/test_files/csv_files.slt`.
I checked it is not vacuous: with this change reverted, that case fails
with the
panic quoted above rather than an error, so the test reproduces the bug.
`cargo fmt --check`, `cargo clippy --all-targets` and the crate's unit
tests are
clean.
## Are there any user-facing changes?
An invalid `null_regex` now produces an error and leaves the session
usable,
where it previously panicked the task. No API changes.
Independent of apache#25254 — that one is the read path in `source.rs`, this
is the
inference path in `file_format.rs` — but they touch the same crate, so
whichever
lands second may want a trivial rebase.
Co-authored-by: Prajwal Narayana <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>1 parent 4dc1c6b commit 6185dee
2 files changed
Lines changed: 27 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
34 | | - | |
| 33 | + | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
540 | 540 | | |
541 | 541 | | |
542 | 542 | | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
543 | 558 | | |
544 | 559 | | |
545 | 560 | | |
| |||
557 | 572 | | |
558 | 573 | | |
559 | 574 | | |
560 | | - | |
561 | | - | |
562 | | - | |
563 | | - | |
| 575 | + | |
| 576 | + | |
564 | 577 | | |
565 | 578 | | |
566 | 579 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
25 | 33 | | |
26 | 34 | | |
27 | 35 | | |
| |||
0 commit comments