Repository navigation
feat: Add expr_adapter_factory getter and with_expr_adapter_factory setter on ListingTable - #27317
Merged
alamb merged 1 commit intoNov 11, 2026
Conversation
…etter on ListingTable
Sruhvx-jpg
force-pushed
the
feat/listing-table-expr-adapter-factory
branch
from
November 10, 2026 20:12
85ba53f to
96a480b
Compare
alamb
approved these changes
Nov 11, 2026
alamb
left a comment
alamb
left a comment
Contributor
There was a problem hiding this comment.
Thanks @Sruhvx-jpg
I wonder if there are other fields we should add accessors for?
Contributor
Author
|
Thanks @alamb! I asked my clanker to perform a research on your concern and this is my review of the research:
I can open separate PRs for |
Contributor
|
Yeah, let's make separate PRs please |
This was referenced Nov 12, 2026
mbutrovich
pushed a commit
to gonappuccino/datafusion
that referenced
this pull request
Nov 12, 2026
…istingTable and ListingTableConfig (apache#27363) ## Which issue does this PR close? Follow-up to apache#27317. Closes apache#27314. ## Rationale for this change In `ListingTable` and `ListingTableConfig`, several fields lacked accessors: 1. `ListingTable::file_schema`: Currently `table.schema()` returns the combined table schema (file schema + Hive partition columns). Callers had no way to inspect or retrieve the raw physical file schema directly, requiring manual partition-column filtering workarounds as seen in apache#27314. 2. `ListingTable::cache`: `with_cache` setter existed, but there was no getter to inspect or forward the attached `FileStatisticsCache`. 3. `ListingTableConfig::expr_adapter_factory`: `with_expr_adapter_factory` setter existed, but the field was `pub(crate)` without a public getter. ## What changes are included in this PR? 1. Added `pub fn file_schema(&self) -> &SchemaRef` to `ListingTable`. 2. Added `pub fn cache(&self) -> Option<&Arc<FileStatisticsCache>>` to `ListingTable`. 3. Added `pub fn expr_adapter_factory(&self) -> Option<&Arc<dyn PhysicalExprAdapterFactory>>` to `ListingTableConfig`. 4. Added unit tests for all three accessors in `table.rs` and `config.rs`. ## Are these changes tested? Yes, tested via unit tests in `datafusion-catalog-listing`: - `test_file_schema` - `test_cache` - `test_config_expr_adapter_factory` Verified with `cargo fmt` and `cargo clippy`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #27314.
Rationale for this change
In
ListingTable, swapping or wrapping thePhysicalExprAdapterFactoryof an existing table currently requires manually unpacking individual fields to construct aListingTableConfigand callingListingTable::try_new(config). This is verbose, discards runtime table state (such as file statistics cache, table definitions, constraints, and column defaults), and redundantly re-evaluates schemas and fingerprints.While
ListingTableConfigprovideswith_expr_adapter_factory,ListingTablewas missing the matching getter and setter afterSchemaAdapterFactorywas replaced byPhysicalExprAdapterFactory.What changes are included in this PR?
pub fn expr_adapter_factory(&self) -> Option<&Arc<dyn PhysicalExprAdapterFactory>>toListingTable.pub fn with_expr_adapter_factory(mut self, expr_adapter_factory: Arc<dyn PhysicalExprAdapterFactory>) -> SelftoListingTable.datafusion/catalog-listing/src/table.rsverifying adapter assignment, retrieval, and replacement on cloned tables without mutating the original table.Are these changes tested?
Yes, tested via
test_with_expr_adapter_factoryindatafusion-catalog-listing. Verified clean formatting (cargo fmt) and zero clippy warnings.