Repository navigation
GH-47728: [Python] Check the source argument in parquet.read_table - #48008
Conversation
raulcd
left a comment
raulcd
left a comment
There was a problem hiding this comment.
The test failure test_read_table_without_dataset failing on the CI jobs is related. It seems a test that was using a MockDataset with a non-existing file is failing with this change, we might have to revisit that test.
Yep, you're right, I've updated it with the new |
raulcd
left a comment
raulcd
left a comment
There was a problem hiding this comment.
LGTM, just a comment about a possible follow up issue.
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 46b033f. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them. |
Rationale for this change
See #47728. Check
sourceargument inpyarrow.parquet.read_tableifpyarrow.datasetis not available.What changes are included in this PR?
Check the
sourceargument, raiseValueErrorif thesourceargument is either a list of.parquetfiles or a directory.Are these changes tested?
Yes
Are there any user-facing changes?
No
In case if the
sourceargument is a directory, I decided not to check it directly, but to catch the exceptions coming from thefs.open_input_file, since it already checks for it, and add extra exception on top of the stack that explains the actual reason.