Skip to content

Commit 1eb7f67

Browse files
alambclaude
andcommitted
Merge apache/main into alamb/pushdown_by_default
Conflicts resolved: - row_filter.rs: main refactored build_row_filter into prebuild_row_filter_candidates / row_filter_from_prebuilt; the filter mask union for one-shot I/O is now computed from the prebuilt candidate list (new prebuilt_filter_mask helper + RowFilterContext::filter_mask) - opener/mod.rs: one_shot_ranges computation rebased onto main's InitialDecoderState block; filter mask sourced from RowFilterContext - push_decoder.rs: PushDecoderStreamState keeps both one_shot_ranges and main's new fully-matched toggle fields - object_store_access progressive_io snapshot updated: main's fully-matched RowFilter skip removes one request for the fully matched row group - dynamic_row_group_pruning.slt: main's new fully-matched suppression test pinned to pushdown_filter_mode=always (it projects only the filter column, which the narrow-projection gate would decline) - dynamic_filter_pushdown_config.slt: expected metrics updated for row_filter_skipped_fully_matched Co-Authored-By: Claude Fable 5 <[email protected]>
2 parents 66962fa + a38bb10 commit 1eb7f67

658 files changed

Lines changed: 42086 additions & 8286 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.ai/skills/audit-datafusion-spark-expression/SKILL.md‎

Lines changed: 996 additions & 0 deletions
Large diffs are not rendered by default.

‎.asf.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ github:
6666
- "cargo check datafusion-proto features"
6767
- "cargo check datafusion features"
6868
- "cargo check datafusion-functions features"
69+
- "cargo check datafusion-spark features"
6970
- "cargo test (amd64)"
7071
- "cargo test datafusion-cli (amd64)"
7172
- "cargo examples (amd64)"

‎.cargo/config.toml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
[alias]
19+
xtask = "run --quiet --locked --package datafusion-xtask --"
20+
ci-step = "run --quiet --locked --package datafusion-xtask -- ci step"

‎.github/dependabot.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,13 @@ updates:
104104
schedule:
105105
interval: "weekly"
106106
labels: [auto-dependencies]
107+
groups:
108+
# all dependency bumps are grouped into a single PR
109+
all-uv-deps:
110+
applies-to: version-updates
111+
patterns:
112+
- "*"
113+
update-types:
114+
- "major"
115+
- "minor"
116+
- "patch"

‎.github/workflows/audit.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ jobs:
4545
steps:
4646
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
4747
- name: Install cargo-audit
48-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
48+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
4949
with:
5050
tool: cargo-audit
5151
- name: Run audit check

‎.github/workflows/breaking_changes_detector.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ jobs:
8989
9090
- name: Install cargo-semver-checks
9191
if: steps.changed_crates.outputs.packages != ''
92-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
92+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
9393
with:
9494
tool: cargo-semver-checks
9595

‎.github/workflows/codeql.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,11 @@ jobs:
4545
persist-credentials: false
4646

4747
- name: Initialize CodeQL
48-
uses: github/codeql-action/init@5595ccaf912efad79be6eef63a5619ff05969be3 # v4
48+
uses: github/codeql-action/init@db488ddef3bf6cb639b32c2e9a7c0a7ea8271d28 # v4
4949
with:
5050
languages: actions
5151

5252
- name: Perform CodeQL Analysis
53-
uses: github/codeql-action/analyze@5595ccaf912efad79be6eef63a5619ff05969be3 # v4
53+
uses: github/codeql-action/analyze@db488ddef3bf6cb639b32c2e9a7c0a7ea8271d28 # v4
5454
with:
5555
category: "/language:actions"

‎.github/workflows/dependencies.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ jobs:
6363
steps:
6464
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
6565
- name: Install cargo-machete
66-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
66+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
6767
with:
6868
6969
- name: Detect unused dependencies

‎.github/workflows/dev.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,9 @@ jobs:
3838
steps:
3939
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
4040
- name: Install HawkEye
41-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
41+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
4242
with:
43-
tool: hawkeye@6.2.0
43+
tool: hawkeye@7.0.0
4444
- name: Run license header check
4545
run: ci/scripts/license_header.sh
4646

@@ -66,7 +66,7 @@ jobs:
6666
source ci/scripts/utils/tool_versions.sh
6767
echo "LYCHEE_VERSION=${LYCHEE_VERSION}" >> "$GITHUB_ENV"
6868
- name: Install lychee
69-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
69+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
7070
with:
7171
tool: lychee@${{ env.LYCHEE_VERSION }}
7272
- name: Run markdown link check
@@ -91,7 +91,7 @@ jobs:
9191
# it may cause checks to fail more often.
9292
# We can upgrade it manually once a while.
9393
- name: Install typos
94-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
94+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
9595
with:
9696
9797
- name: Run typos check

‎.github/workflows/docs.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ jobs:
4343
path: asf-site
4444

4545
- name: Setup uv
46-
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
46+
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
4747

4848
- name: Install dependencies
4949
run: uv sync --package datafusion-docs
@@ -53,7 +53,7 @@ jobs:
5353
sudo apt-get update
5454
sudo apt-get install -y graphviz
5555
- name: Install cargo-depgraph
56-
uses: taiki-e/install-action@7f4eb899022d8fe70b20c4f3de697aa85c309026 # v2.85.11
56+
uses: taiki-e/install-action@ba47c86ac325773530516bb756137ac718732518 # v2.86.5
5757
with:
5858
5959

0 commit comments

Comments
 (0)