Repository navigation
Commit 1eb7f67
Merge apache/main into alamb/pushdown_by_default
Conflicts resolved:
- row_filter.rs: main refactored build_row_filter into
prebuild_row_filter_candidates / row_filter_from_prebuilt; the filter
mask union for one-shot I/O is now computed from the prebuilt
candidate list (new prebuilt_filter_mask helper +
RowFilterContext::filter_mask)
- opener/mod.rs: one_shot_ranges computation rebased onto main's
InitialDecoderState block; filter mask sourced from RowFilterContext
- push_decoder.rs: PushDecoderStreamState keeps both one_shot_ranges and
main's new fully-matched toggle fields
- object_store_access progressive_io snapshot updated: main's
fully-matched RowFilter skip removes one request for the fully
matched row group
- dynamic_row_group_pruning.slt: main's new fully-matched suppression
test pinned to pushdown_filter_mode=always (it projects only the
filter column, which the narrow-projection gate would decline)
- dynamic_filter_pushdown_config.slt: expected metrics updated for
row_filter_skipped_fully_matched
Co-Authored-By: Claude Fable 5 <[email protected]>658 files changed
Lines changed: 42086 additions & 8286 deletions
File tree
- .ai/skills/audit-datafusion-spark-expression
- .cargo
- .github
- workflows
- benchmarks
- benches
- sql_benchmarks
- clickbench_extended/benchmarks
- h2o/benchmarks/window
- smj/benchmarks
- src
- bin
- ci/scripts
- datafusion-cli
- src
- tests
- datafusion-examples
- examples
- custom_data_source
- data_io
- proto
- statistics
- udf
- datafusion
- catalog-listing/src
- catalog/src
- memory
- common
- benches
- src
- hash_utils
- scalar
- utils
- core
- benches
- src
- bin
- dataframe
- datasource
- file_format
- listing
- physical_plan
- execution
- context
- test_util
- tests
- custom_sources_cases
- dataframe
- datasource
- data
- execution
- fuzz_cases
- memory_limit
- memory_limit_validation
- parquet
- physical_optimizer
- sql
- aggregates
- tracing
- user_defined
- datasource-arrow/src
- datasource-avro/src
- datasource-csv/src
- datasource-json/src
- datasource-parquet
- benches
- src
- opener
- datasource/src
- file_scan_config
- file_stream
- doc/src
- execution/src
- cache
- expr-common
- src
- expr/src
- expr_rewriter
- logical_plan
- ffi
- src
- expr
- session
- tests
- udaf
- udwf
- tests
- functions-aggregate-common/src
- aggregate
- avg_distinct
- groups_accumulator
- functions-aggregate
- benches
- src
- min_max
- functions-nested/src
- functions-table/src
- functions-window/src
- functions
- benches
- datetime_expressions
- math_expressions
- misc
- regex_expressions
- string_expressions
- unicode_expressions
- src
- core
- crypto
- datetime
- encoding
- math
- string
- unicode
- optimizer/src
- analyzer
- optimize_projections
- simplify_expressions
- physical-expr-adapter/src
- physical-expr-common/src
- metrics
- physical-expr
- benches
- src
- equivalence
- properties
- expressions
- binary
- case/literal_lookup_table
- dynamic_filters
- in_list
- simplifier
- window
- physical-optimizer/src
- ensure_requirements/enforce_sorting
- physical-plan
- benches
- src
- aggregates
- aggregate_hash_table
- group_values
- multi_group_by
- joins
- hash_join
- piecewise_merge_join
- sort_merge_join
- operator_statistics
- repartition
- sorts
- spill
- topk
- windows
- proto-common/gen/src
- proto-models
- gen/src
- proto
- src/generated
- proto
- src/physical_plan
- tests/cases
- plans
- pruning
- benches
- src
- session/src
- spark
- src/function
- bitmap
- hash
- math
- string
- url
- sqllogictest
- bin
- src
- engines
- datafusion_engine
- postgres_engine
- test_context
- test_files
- array
- spark
- math
- string
- sql
- src
- expr
- relation
- unparser
- tests
- cases
- substrait
- src/logical_plan
- consumer/expr
- producer
- expr
- tests/cases
- dev
- changelog
- depcheck
- release
- docs
- scripts
- source
- library-user-guide
- upgrading
- user-guide
- cli
- sql
- test-utils/src
- array_gen
- xtask
- src
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| 69 | + | |
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
| 48 | + | |
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | | - | |
| 92 | + | |
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
| 48 | + | |
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
53 | | - | |
| 53 | + | |
54 | 54 | | |
55 | 55 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
66 | | - | |
| 66 | + | |
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
94 | | - | |
| 94 | + | |
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
| 56 | + | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| |||
0 commit comments