Skip to content

Commit ceb91b5

Browse files
jpascucci-nvrwgk
jpascucci-nv
and
rwgk
authored
CI: use shallow, blob-filtered CCCL submodule checkouts (#3392)
* CI: use shallow, blob-filtered CCCL submodule checkouts closes #3351 actions/checkout applies its blob filter only to the root repository, leaving CCCL submodule clones unfiltered and causing slow, variable checkout times. Fetch submodules separately with --depth=1 --filter=blob:none across build, source-test, coverage, and rendered-documentation jobs. Preserve full root history and release tags for setuptools-scm, plus URL syncing and forced checkout for reused workspaces. Validation: fresh and reused checkout checks passed; nine workflow regression tests passed. Actionlint diagnostics are unchanged. Windows checkout timing remains unverified. * docs: use filtered shallow CCCL checkouts for contributors — codex GPT-6 on behalf of Ralf — Preserve root history and release tags while fetching CCCL separately with shallow history and blob filtering. Align the contributor guide and package installation examples, and control submodule fetches explicitly on updates. Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> * docs: simplify filtered clone instructions — codex GPT-6 on behalf of Ralf — Use Git's native submodule filtering in each clone example and retain the ordinary update instructions. Preserve full commit history and release tags while keeping the filtered recipe directly visible to users and agents. Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> --------- Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
1 parent ddc2a2a commit ceb91b5

10 files changed

Lines changed: 93 additions & 22 deletions

File tree

‎.github/workflows/build-wheel.yml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,13 @@ jobs:
7272
# but not historical blobs.
7373
fetch-depth: 0
7474
filter: blob:none
75-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
76-
submodules: true
75+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
76+
submodules: false
77+
78+
- name: Checkout CCCL submodule
79+
run: |
80+
git submodule sync --recursive
81+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
7782
7883
- name: Register problem matchers
7984
# Compiler and Cython errors from the wheel builds become annotations

‎.github/workflows/ci-pixi-source-test.yml‎

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -88,8 +88,13 @@ jobs:
8888
# cuda.core's "cuda.bindings 12.x or 13.x must be installed" guard.
8989
fetch-depth: 0
9090
filter: blob:none
91-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
92-
submodules: true
91+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
92+
submodules: false
93+
94+
- name: Checkout CCCL submodule
95+
run: |
96+
git submodule sync --recursive
97+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
9398
9499
- name: Setup pixi
95100
uses: ./.github/actions/setup-pixi
@@ -147,8 +152,13 @@ jobs:
147152
# cuda.core's "cuda.bindings 12.x or 13.x must be installed" guard.
148153
fetch-depth: 0
149154
filter: blob:none
150-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
151-
submodules: true
155+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
156+
submodules: false
157+
158+
- name: Checkout CCCL submodule
159+
run: |
160+
git submodule sync --recursive
161+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
152162
153163
- name: Setup pixi
154164
uses: ./.github/actions/setup-pixi
@@ -215,8 +225,13 @@ jobs:
215225
# cuda.core's "cuda.bindings 12.x or 13.x must be installed" guard.
216226
fetch-depth: 0
217227
filter: blob:none
218-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
219-
submodules: true
228+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
229+
submodules: false
230+
231+
- name: Checkout CCCL submodule
232+
run: |
233+
git submodule sync --recursive
234+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
220235
221236
- name: Setup proxy cache
222237
uses: nv-gha-runners/setup-proxy-cache@main

‎.github/workflows/coverage.yml‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,13 @@ jobs:
7171
# but not historical blobs.
7272
fetch-depth: 0
7373
filter: blob:none
74-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
75-
submodules: true
74+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
75+
submodules: false
76+
77+
- name: Checkout CCCL submodule
78+
run: |
79+
git submodule sync --recursive
80+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
7681
7782
- name: Fix workspace ownership
7883
run: |
@@ -238,8 +243,13 @@ jobs:
238243
# but not historical blobs.
239244
fetch-depth: 0
240245
filter: blob:none
241-
# third_party/cccl provides the CCCL headers cuda.core compiles against.
242-
submodules: true
246+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
247+
submodules: false
248+
249+
- name: Checkout CCCL submodule
250+
run: |
251+
git submodule sync --recursive
252+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
243253
244254
- name: Set up Python
245255
uses: $/.github/actions/setup-hardened-python

‎.github/workflows/lychee.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,14 @@ jobs:
6363
# Keep SCM history/tags without downloading historical source blobs.
6464
filter: blob:none
6565
persist-credentials: false
66-
# Rendered docs build cuda.core against the pinned CCCL headers.
67-
submodules: ${{ matrix.kind == 'rendered' }}
66+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
67+
submodules: false
68+
69+
- name: Checkout CCCL submodule
70+
if: ${{ matrix.kind == 'rendered' }}
71+
run: |
72+
git submodule sync --recursive
73+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
6874
6975
- name: Set up Python
7076
id: python

‎.github/workflows/test-sdist-linux.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,13 @@ jobs:
4747
# the commit graph (`git describe`) but not historical blobs.
4848
fetch-depth: 0
4949
filter: blob:none
50-
submodules: true
50+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
51+
submodules: false
52+
53+
- name: Checkout CCCL submodule
54+
run: |
55+
git submodule sync --recursive
56+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
5157
5258
- name: Set up Python
5359
uses: ./.github/actions/setup-hardened-python

‎.github/workflows/test-sdist-windows.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,13 @@ jobs:
5656
# the commit graph (`git describe`) but not historical blobs.
5757
fetch-depth: 0
5858
filter: blob:none
59-
submodules: true
59+
# Fetch CCCL separately: checkout's filter does not apply to submodules.
60+
submodules: false
61+
62+
- name: Checkout CCCL submodule
63+
run: |
64+
git submodule sync --recursive
65+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
6066
6167
- name: Set up Python
6268
uses: ./.github/actions/setup-hardened-python

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,10 +69,11 @@ ran and any validation gaps.
6969
including the Developer Mode registry check.** Complete that setup before
7070
running the clone command.
7171

72-
From the directory where you want to create the checkout, run:
72+
With Git 2.36 or newer, run this from the directory where you want to
73+
create the checkout:
7374

7475
```console
75-
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules https://github.com/NVIDIA/cuda-python.git
76+
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules --filter=blob:none --also-filter-submodules https://github.com/NVIDIA/cuda-python.git
7677
$ cd cuda-python
7778
```
7879

@@ -81,6 +82,9 @@ This recipe sets up the repository for source builds and development:
8182
- `core.symlinks=true` materializes symlinks during checkout, including the
8283
build-critical `cuda_core/_build_shared.py`.
8384
- `--recurse-submodules` initializes the CCCL headers needed to build `cuda.core`.
85+
- `--filter=blob:none --also-filter-submodules` skips historical file
86+
contents in the repository and CCCL. Git downloads those contents
87+
when needed; commit history and release tags remain available.
8488
- `submodule.recurse=true` keeps initialized submodules in sync on `git pull`,
8589
`git checkout`, and `git switch`.
8690
- The clone retains history and release tags for correct package versions.

‎ci/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,25 @@ The pinned checkout action's
2424
[`README`](https://github.com/actions/checkout/blob/3d3c42e5aac5ba805825da76410c181273ba90b1/README.md#fetch-all-history-for-all-tags-and-branches)
2525
documents `fetch-depth: 0` for fetching all history, branches, and tags.
2626

27+
## CCCL Submodule Checkout
28+
29+
Jobs that build `cuda.core` fetch its pinned CCCL headers in a separate step:
30+
set `submodules: false` on `actions/checkout`, then run:
31+
32+
```bash
33+
git submodule sync --recursive
34+
git submodule update --init --force --depth=1 --filter=blob:none --recursive
35+
```
36+
37+
The checkout action's `filter` input applies only to the root repository, so
38+
letting it fetch submodules downloads CCCL's unfiltered history. The separate
39+
step limits submodule history to one commit and skips historical blobs, while
40+
checking out the exact revision pinned by the root repository. Syncing URLs
41+
and forcing checkout preserve the action's behavior on reused workspaces.
42+
Keep `fetch-depth: 0` and `filter: blob:none` on the root checkout: package
43+
versioning still needs its complete ancestry and release tags. See
44+
[issue #3351](https://github.com/NVIDIA/cuda-python/issues/3351).
45+
2746
## Documentation Link Checks
2847

2948
Two pre-commit hooks check links with one pinned native wheel (`lychee-bin`,

‎cuda_core/docs/source/install.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ recipe below. On native Windows outside WSL, complete its Windows setup
130130

131131
.. code-block:: console
132132
133-
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules https://github.com/NVIDIA/cuda-python.git
133+
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules --filter=blob:none --also-filter-submodules https://github.com/NVIDIA/cuda-python.git
134134
$ cd cuda-python/cuda_core
135135
$ pip install .
136136
@@ -161,7 +161,7 @@ For example, to work on ``cuda-core`` against CUDA 13:
161161

162162
.. code-block:: console
163163
164-
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules https://github.com/NVIDIA/cuda-python.git
164+
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules --filter=blob:none --also-filter-submodules https://github.com/NVIDIA/cuda-python.git
165165
$ cd cuda-python/cuda_core
166166
$ uv venv
167167
$ source .venv/bin/activate # On Windows: .venv\Scripts\activate
@@ -184,7 +184,7 @@ For example, to work against CUDA 13:
184184

185185
.. code-block:: console
186186
187-
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules https://github.com/NVIDIA/cuda-python.git
187+
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules --filter=blob:none --also-filter-submodules https://github.com/NVIDIA/cuda-python.git
188188
$ cd cuda-python
189189
$ pixi run -e cu13 test-core
190190

‎cuda_pathfinder/docs/source/install.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ before cloning.
6464

6565
.. code-block:: console
6666
67-
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules https://github.com/NVIDIA/cuda-python.git
67+
$ git clone --config core.symlinks=true --config submodule.recurse=true --recurse-submodules --filter=blob:none --also-filter-submodules https://github.com/NVIDIA/cuda-python.git
6868
$ cd cuda-python/cuda_pathfinder
6969
$ pip install .
7070

0 commit comments

Comments
 (0)