Skip to content

Commit 9c00024

Browse files
kumarUjjawalOmega359
authored andcommitted
dev: include dependency checks in the local lint suite (apache#25399)
## Which issue does this PR close? - Part of apache#21048. ## Rationale for this change `dev/rust_lint.sh` does not run the two dependency checks from the "Dependencies" workflow. A contributor finds a dependency cycle or an unused dependency only in CI. ## What changes are included in this PR? - `ci/scripts/check_circular_dependencies.sh` runs `cargo run --locked` in `dev/depcheck`. - `ci/scripts/check_unused_dependencies.sh` runs `cargo machete --with-metadata` from the repository root. - The workflow calls both scripts. `CARGO_MACHETE_VERSION="0.9"` moves to `ci/scripts/utils/tool_versions.sh`, and the workflow loads it from there. - `dev/rust_lint.sh` installs a missing `cargo-machete` and runs both scripts as read-only steps. - `docs/source/contributor-guide/testing.md` documents the standalone commands. ## What is the testing strategy for this PR? - In a clean clone, each original command and its wrapper gave identical output with exit 0, and no lockfile changed. - A fixture with stubbed steps and tools covered the working directories, exit codes, install path, and fail-fast behavior. - The parsed workflow YAML matches `main` except the changed steps. The full `./dev/rust_lint.sh` passes. ## Are there any user-facing changes? No.
1 parent a94f698 commit 9c00024

6 files changed

Lines changed: 104 additions & 5 deletions

File tree

‎.github/workflows/dependencies.yml‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,9 +51,7 @@ jobs:
5151
with:
5252
rust-version: stable
5353
- name: Check dependencies
54-
working-directory: dev/depcheck
55-
run: |
56-
cargo run --locked
54+
run: bash ci/scripts/check_circular_dependencies.sh
5755

5856
detect-unused-dependencies:
5957
name: Detect Unused Dependencies
@@ -62,9 +60,13 @@ jobs:
6260
image: amd64/rust
6361
steps:
6462
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
63+
- name: Load tool versions
64+
run: |
65+
. ci/scripts/utils/tool_versions.sh
66+
echo "CARGO_MACHETE_VERSION=${CARGO_MACHETE_VERSION}" >> "$GITHUB_ENV"
6567
- name: Install cargo-machete
6668
uses: taiki-e/install-action@3f74d7c16a4242f1c95561e98edc25d36adb4375 # v2.87.12
6769
with:
68-
tool: cargo-machete@0.9
70+
tool: cargo-machete@${{ env.CARGO_MACHETE_VERSION }}
6971
- name: Detect unused dependencies
70-
run: cargo machete --with-metadata
72+
run: bash ci/scripts/check_unused_dependencies.sh
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# Checks for circular dependencies between DataFusion crates with
21+
# `dev/depcheck`, the same way the "Circular Dependency Check" job does.
22+
23+
set -euo pipefail
24+
25+
SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
26+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
27+
ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
28+
29+
# depcheck is outside the root workspace and finds the root manifest from its
30+
# own directory, so run Cargo from there.
31+
cd "${ROOT_DIR}/dev/depcheck"
32+
33+
echo "[${SCRIPT_NAME}] \`cargo run --locked\` in dev/depcheck"
34+
cargo run --locked
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# Detects unused dependencies with `cargo machete`, the same way the
21+
# "Detect Unused Dependencies" job does.
22+
23+
set -euo pipefail
24+
25+
SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
26+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
27+
ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
28+
29+
source "${SCRIPT_DIR}/utils/tool_versions.sh"
30+
31+
if ! command -v cargo-machete &> /dev/null; then
32+
echo "[${SCRIPT_NAME}] cargo-machete is required. Install it with: cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}" >&2
33+
exit 1
34+
fi
35+
36+
cd "${ROOT_DIR}"
37+
38+
echo "[${SCRIPT_NAME}] \`cargo machete --with-metadata\`"
39+
cargo machete --with-metadata

‎ci/scripts/utils/tool_versions.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,4 @@
2222

2323
PRETTIER_VERSION="2.7.1"
2424
LYCHEE_VERSION="0.23.0"
25+
CARGO_MACHETE_VERSION="0.9"

‎dev/rust_lint.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,7 @@ ensure_tool "hawkeye" "cargo install hawkeye --locked"
113113
ensure_tool "typos" "cargo install typos-cli --locked"
114114
ensure_tool "lychee" "cargo install lychee --locked --version ${LYCHEE_VERSION}"
115115
ensure_tool "cargo-audit" "cargo install cargo-audit --locked"
116+
ensure_tool "cargo-machete" "cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}"
116117

117118
run_step() {
118119
local name="$1"
@@ -135,6 +136,8 @@ declare -a READONLY_STEPS=(
135136
"ci/scripts/check_asf_yaml_status_checks.py|false"
136137
"ci/scripts/markdown_link_check.sh|false"
137138
"ci/scripts/security_audit.sh|false"
139+
"ci/scripts/check_circular_dependencies.sh|false"
140+
"ci/scripts/check_unused_dependencies.sh|false"
138141
"ci/scripts/rust_docs.sh|false"
139142
)
140143

‎docs/source/contributor-guide/testing.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,26 @@ The audit fetches the RustSec advisory database. A new advisory or a different
254254

255255
[cargo-audit]: https://github.com/rustsec/rustsec/blob/main/cargo-audit/README.md
256256

257+
## Dependency Checks
258+
259+
CI runs two dependency checks, and `./dev/rust_lint.sh` runs both:
260+
261+
- `ci/scripts/check_circular_dependencies.sh` builds and runs [`dev/depcheck`],
262+
which fails on dependency cycles between DataFusion crates.
263+
- `ci/scripts/check_unused_dependencies.sh` runs `cargo machete --with-metadata`
264+
from the repository root. The lint suite installs [cargo-machete] with the
265+
version in `ci/scripts/utils/tool_versions.sh` if it is missing.
266+
267+
To run either check on its own:
268+
269+
```shell
270+
./ci/scripts/check_circular_dependencies.sh
271+
./ci/scripts/check_unused_dependencies.sh
272+
```
273+
274+
[`dev/depcheck`]: https://github.com/apache/datafusion/tree/main/dev/depcheck
275+
[cargo-machete]: https://github.com/bnjbvr/cargo-machete
276+
257277
## Benchmarks
258278

259279
### Criterion Benchmarks

0 commit comments

Comments
 (0)