Skip to content

Commit 659b475

Browse files
authored
chore: Skip datafusion-sqllogictest in semver check (apache#26577)
## Which issue does this PR close? - Related to apache#25088, apache#25148 ## Rationale for this change Many PRs modify some Rust code and also touch some SLT files. At present, this results in checking all of `datafusion-sqllogictest` for semver compatibility, which is very expensive (because it depends on almost all of the other crates). `datafusion-sqllogictest` is a published crate but it is test tooling that isn't part of the library API, so I think skipping semver checks for it is not unreasonable. The longer-term goal here is to make the semver check cheap enough that we can run it using `ubuntu-slim`, but we're probably not at that point yet. In any case, reducing semver check overhead is a useful improvement by itself. ## What changes are included in this PR? See above. We also use `install-action` to install `protoc` rather than `apt`, since it's much faster. ## What is the testing strategy for this PR? CI change only. ## Are there any user-facing changes? No.
1 parent 95ef315 commit 659b475

2 files changed

Lines changed: 17 additions & 12 deletions

File tree

‎.github/workflows/breaking_changes_detector.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@
1818
# Detect semver-incompatible (breaking) API changes in crates modified by a PR.
1919
#
2020
# Only public workspace crates that have file changes are checked.
21-
# Internal crates (benchmarks, test-utils, sqllogictest, doc) are excluded.
21+
# Crates with `publish = false` (such as benchmarks and test-utils) and
22+
# datafusion-sqllogictest are excluded.
2223
#
2324
# This workflow only runs cargo-semver-checks and uploads the result as an
2425
# artifact. The actual PR comment is posted by a companion workflow
@@ -78,14 +79,13 @@ jobs:
7879
echo "packages=$PACKAGES" >> "$GITHUB_OUTPUT"
7980
echo "Changed crates: $PACKAGES"
8081
81-
# `datafusion-substrait` (and crates that depend on it via sqllogictest)
82-
# have a build script that calls protoc, which is not preinstalled on
83-
# ubuntu-latest runners.
84-
- name: Install Protobuf Compiler
82+
# The `substrait` crate's build script runs protoc, which is not
83+
# preinstalled on ubuntu-latest runners.
84+
- name: Install protoc
8585
if: steps.changed_crates.outputs.packages != ''
86-
run: |
87-
sudo apt-get update
88-
sudo apt-get install -y protobuf-compiler
86+
uses: taiki-e/install-action@8cca0918f8471c33515202b73f56ab27f2bed283 # v2.87.29
87+
with:
88+
8989

9090
- name: Install cargo-semver-checks
9191
if: steps.changed_crates.outputs.packages != ''

‎ci/scripts/changed_crates.sh‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,12 @@
2121
# Subcommands:
2222
# changed-crates <base_ref>
2323
# Print space-separated list of crate names whose files changed vs base_ref.
24-
# Only published workspace members (those without `publish = false`) are
25-
# considered.
24+
# Only published workspace members (those without `publish = false`)
25+
# are considered. We also omit datafusion-sqllogictest. It is published
26+
# so that other projects can write tests with DataFusion's sqllogictest
27+
# driver, but it is test tooling rather than part of DataFusion's
28+
# library API. It also depends on nearly every other DataFusion crate,
29+
# so checking it usually takes most of the job's time.
2630
#
2731
# semver-check <base_ref> <packages...>
2832
# Run cargo-semver-checks for the given packages against base_ref.
@@ -42,12 +46,13 @@ cmd_changed_crates() {
4246
# 2. Every publishable workspace member, one per line as
4347
# "<crate-name> <crate-dir>". `publish = false` in Cargo.toml shows
4448
# up as `"publish": []` in cargo metadata, so filtering on that
45-
# excludes internal crates without a manual exclusion list.
49+
# excludes unpublished crates. datafusion-sqllogictest is excluded by
50+
# name; see the comment at the top of this file.
4651
local crates
4752
crates=$(cargo metadata --no-deps --format-version 1 | jq -r '
4853
(.workspace_root + "/") as $root
4954
| .packages[]
50-
| select(.publish != [])
55+
| select(.publish != [] and .name != "datafusion-sqllogictest")
5156
| "\(.name) \(.manifest_path | ltrimstr($root) | rtrimstr("/Cargo.toml"))"
5257
')
5358

0 commit comments

Comments
 (0)