Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4aadce2
Add nightly CI pipeline for optional-dependency testing (PyTorch, num…
leofang Apr 29, 2026
6dea4c3
Add concurrency group and fix checks job in ci-nightly.yml
leofang Apr 29, 2026
ac5238c
Temporarily add push trigger to ci-nightly.yml for testing
leofang Apr 29, 2026
cb5aefa
Use shallow clone (fetch-depth: 1) in ci-nightly.yml
leofang Apr 29, 2026
9286598
Fix artifact name mismatch in nightly CI by passing source SHA
leofang Apr 29, 2026
0b7cc50
Fix nightly CI: single pip call, display step, numba cmd, Windows VC++
leofang Apr 29, 2026
edeaa76
Match CUDA_VER to TORCH_CUDA in nightly pytorch matrix
leofang Apr 29, 2026
6f04205
Match CUDA_VER to TORCH_CUDA in nightly pytorch matrix
leofang Apr 29, 2026
da50070
Merge remote-tracking branch 'origin/main' into ci-nightly-optdeps
leofang Apr 30, 2026
cec1d8d
Remove numba-cuda test_linker.py patch (fixed upstream)
leofang Apr 30, 2026
833490e
Merge branch 'ci-nightly-optdeps' of https://github.com/leofang/cuda-…
leofang Apr 30, 2026
2a749d5
Merge branch 'main' into ci-nightly-optdeps
leofang May 1, 2026
bb75dcc
Merge branch 'main' into ci-nightly-optdeps
leofang May 11, 2026
dd909d3
Merge remote-tracking branch 'origin/main' into ci-nightly-optdeps
leofang May 14, 2026
ad2a093
Merge branch 'ci-nightly-optdeps' of https://github.com/leofang/cuda-…
leofang May 14, 2026
cd4277a
Consolidate CI run ID lookups into ci/tools/lookup-run-id
leofang May 14, 2026
901dc09
Remove cuda_pathfinder listing from Windows display step
leofang May 14, 2026
b42a4be
Enforce --tag/--branch mutual exclusivity in lookup-run-id
leofang May 14, 2026
0415bab
Show torch version and numba-cuda in nightly CI job names
leofang May 14, 2026
025a26e
Improve CI job name format with descriptive labels
leofang May 14, 2026
a566781
Handle BINDINGS_SOURCE in nightly install and display steps
leofang May 14, 2026
9fd2f3e
Fix display step guard: skip only for published bindings
leofang May 14, 2026
a232927
Pin nightly pytorch 'latest' to 2.11 (tensor bridge version cap)
leofang May 14, 2026
7d8c286
Use full version 2.11.0 for torch pin
leofang May 14, 2026
2396430
Add arm64 pytorch nightly entries and fix incorrect comment
leofang May 14, 2026
4d1c871
Address rwgk review: cron offset, CUDA_BUILD_VER from source SHA, che…
leofang May 15, 2026
7d36fea
Consolidate core and nightly-* install logic in run-tests
leofang May 15, 2026
f88eddd
revert the test trigger
leofang May 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add nightly CI pipeline for optional-dependency testing (PyTorch, num…
…ba-cuda)

Add ci-nightly.yml that downloads wheels from the latest successful CI
run on main and tests them against PyTorch and numba-cuda, without
rebuilding.

Key changes:
- ci-nightly.yml: new orchestrator (schedule 2 AM UTC + workflow_dispatch)
- test-wheel-linux/windows.yml: add run-id input for cross-run artifact
  downloads, and test-mode input (standard/nightly-pytorch/nightly-numba-cuda)
  with conditional test steps
- ci/test-matrix.yml: add nightly entries with MODE field (4 pytorch +
  6 numba-cuda across linux-64, linux-aarch64, win-64)
- ci/tools/run-tests: add nightly-install mode that installs all wheels
  without running standard tests

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
  • Loading branch information
leofang and claude committed Apr 29, 2026
commit 4aadce2b6b0933557c6831eb37dd3cf9419bf47c
177 changes: 177 additions & 0 deletions .github/workflows/ci-nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# SPDX-License-Identifier: Apache-2.0

# Nightly CI pipeline that tests optional dependencies (PyTorch, numba-cuda)
# against the latest cuda-python wheels built on main.
#
# This workflow does NOT build wheels — it downloads them from the latest
# successful CI run on main and runs integration tests with optional deps.

name: "CI: Nightly optional-deps"

on:
schedule:
# 2 AM UTC daily, after the midnight main CI build finishes
- cron: "0 2 * * *"
Comment thread
leofang marked this conversation as resolved.
Resolved
workflow_dispatch:
inputs:
run-id:
description: >
Override the CI run ID to download artifacts from.
Leave empty to auto-detect the latest successful main run.
type: string
default: ''

jobs:
find-wheels:
runs-on: ubuntu-latest
outputs:
RUN_ID: ${{ steps.find.outputs.run_id }}
CUDA_BUILD_VER: ${{ steps.get-vars.outputs.cuda_build_ver }}
Comment thread
leofang marked this conversation as resolved.
Resolved
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0

- name: Get CUDA build versions
id: get-vars
run: |
cuda_build_ver=$(yq '.cuda.build.version' ci/versions.yml)
echo "cuda_build_ver=$cuda_build_ver" >> $GITHUB_OUTPUT

- name: Find latest successful CI run on main
id: find
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if [[ -n "${{ inputs.run-id }}" ]]; then
echo "run_id=${{ inputs.run-id }}" >> $GITHUB_OUTPUT
echo "Using manually specified run ID: ${{ inputs.run-id }}"
exit 0
fi

RUN_ID=$(gh run list \
-b main \
-L 1 \
-w "CI" \
-s success \
-R "${{ github.repository }}" \
--json databaseId \
| jq -r '.[0].databaseId')

if [[ -z "$RUN_ID" || "$RUN_ID" == "null" ]]; then
echo "::error::No successful CI run found on main"
exit 1
fi

echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT
echo "Using latest successful CI run: $RUN_ID"

# ── PyTorch interop tests ──

test-pytorch-linux:
name: "Nightly PyTorch (linux-64)"
if: ${{ github.repository_owner == 'nvidia' }}
needs: find-wheels
permissions:
contents: read
actions: read
secrets: inherit
uses: ./.github/workflows/test-wheel-linux.yml
with:
build-type: nightly
host-platform: linux-64
build-ctk-ver: ${{ needs.find-wheels.outputs.CUDA_BUILD_VER }}
run-id: ${{ needs.find-wheels.outputs.RUN_ID }}
test-mode: nightly-pytorch
matrix_filter: 'map(select(.MODE == "nightly-pytorch"))'

test-pytorch-windows:
name: "Nightly PyTorch (win-64)"
if: ${{ github.repository_owner == 'nvidia' }}
needs: find-wheels
permissions:
contents: read
actions: read
secrets: inherit
uses: ./.github/workflows/test-wheel-windows.yml
with:
build-type: nightly
host-platform: win-64
build-ctk-ver: ${{ needs.find-wheels.outputs.CUDA_BUILD_VER }}
run-id: ${{ needs.find-wheels.outputs.RUN_ID }}
test-mode: nightly-pytorch
matrix_filter: 'map(select(.MODE == "nightly-pytorch"))'

# ── numba-cuda tests ──

test-numba-cuda-linux-64:
name: "Nightly numba-cuda (linux-64)"
if: ${{ github.repository_owner == 'nvidia' }}
needs: find-wheels
permissions:
contents: read
actions: read
secrets: inherit
uses: ./.github/workflows/test-wheel-linux.yml
with:
build-type: nightly
host-platform: linux-64
build-ctk-ver: ${{ needs.find-wheels.outputs.CUDA_BUILD_VER }}
run-id: ${{ needs.find-wheels.outputs.RUN_ID }}
test-mode: nightly-numba-cuda
matrix_filter: 'map(select(.MODE == "nightly-numba-cuda"))'

test-numba-cuda-linux-aarch64:
name: "Nightly numba-cuda (linux-aarch64)"
if: ${{ github.repository_owner == 'nvidia' }}
needs: find-wheels
permissions:
contents: read
actions: read
secrets: inherit
uses: ./.github/workflows/test-wheel-linux.yml
with:
build-type: nightly
host-platform: linux-aarch64
build-ctk-ver: ${{ needs.find-wheels.outputs.CUDA_BUILD_VER }}
run-id: ${{ needs.find-wheels.outputs.RUN_ID }}
test-mode: nightly-numba-cuda
matrix_filter: 'map(select(.MODE == "nightly-numba-cuda"))'

test-numba-cuda-windows:
name: "Nightly numba-cuda (win-64)"
if: ${{ github.repository_owner == 'nvidia' }}
needs: find-wheels
permissions:
contents: read
actions: read
secrets: inherit
uses: ./.github/workflows/test-wheel-windows.yml
with:
build-type: nightly
host-platform: win-64
build-ctk-ver: ${{ needs.find-wheels.outputs.CUDA_BUILD_VER }}
run-id: ${{ needs.find-wheels.outputs.RUN_ID }}
test-mode: nightly-numba-cuda
matrix_filter: 'map(select(.MODE == "nightly-numba-cuda"))'

# ── Status check ──

checks:
name: Nightly check status
if: always()
runs-on: ubuntu-latest
needs:
Comment thread
leofang marked this conversation as resolved.
Resolved
- test-pytorch-linux
- test-pytorch-windows
- test-numba-cuda-linux-64
- test-numba-cuda-linux-aarch64
- test-numba-cuda-windows
steps:
- name: Exit
run: |
# if any dependencies were cancelled or failed, that's a failure
jq -e 'all(.[]; . == "success" or . == "skipped")' <<< '${{ toJSON(needs.*.result) }}'
73 changes: 71 additions & 2 deletions .github/workflows/test-wheel-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,18 @@ on:
skip-bindings-test:
Comment thread
rwgk marked this conversation as resolved.
Resolved
type: boolean
default: false
run-id:
description: >
Workflow run ID to download artifacts from.
Defaults to the current run when empty.
type: string
default: ''
test-mode:
description: >
Test mode: 'standard' (default), 'nightly-pytorch', or
'nightly-numba-cuda'.
type: string
default: 'standard'

defaults:
run:
Expand Down Expand Up @@ -128,20 +140,26 @@ jobs:
with:
name: cuda-pathfinder-wheel
path: ./cuda_pathfinder
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Download cuda-python build artifacts
if: ${{ env.USE_BACKPORT_BINDINGS == '0' }}
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: cuda-python-wheel
path: .
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Download cuda.bindings build artifacts
if: ${{ env.USE_BACKPORT_BINDINGS == '0' }}
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ env.CUDA_BINDINGS_ARTIFACT_NAME }}
path: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Download cuda-python & cuda.bindings build artifacts from the prior branch
if: ${{ env.USE_BACKPORT_BINDINGS == '1' }}
Expand Down Expand Up @@ -194,6 +212,8 @@ jobs:
with:
name: ${{ env.CUDA_BINDINGS_ARTIFACT_NAME }}-tests
path: ${{ env.CUDA_BINDINGS_CYTHON_TESTS_DIR }}
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Display structure of downloaded cuda.bindings Cython tests
if: ${{ env.SKIP_CYTHON_TEST == '0' }}
Expand All @@ -206,6 +226,8 @@ jobs:
with:
name: ${{ env.CUDA_CORE_ARTIFACT_NAME }}
path: ${{ env.CUDA_CORE_ARTIFACTS_DIR }}
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Display structure of downloaded cuda.core build artifacts
run: |
Expand All @@ -218,6 +240,8 @@ jobs:
with:
name: ${{ env.CUDA_CORE_ARTIFACT_NAME }}-tests
path: ${{ env.CUDA_CORE_CYTHON_TESTS_DIR }}
run-id: ${{ inputs.run-id || github.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Display structure of downloaded cuda.core Cython tests
if: ${{ env.SKIP_CYTHON_TEST == '0' }}
Expand Down Expand Up @@ -256,35 +280,39 @@ jobs:
- name: Set up test repetition on nightly runs
run: echo "PYTEST_ADDOPTS=\"--count=${{ inputs.nruns }}\"" >> "$GITHUB_ENV"

# ── Standard test steps (skipped for nightly modes) ──
- name: Run cuda.pathfinder tests with see_what_works
if: ${{ inputs.test-mode == 'standard' }}
env:
CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS: see_what_works
CUDA_PATHFINDER_TEST_FIND_NVIDIA_HEADERS_STRICTNESS: see_what_works
CUDA_PATHFINDER_TEST_FIND_NVIDIA_BITCODE_LIB_STRICTNESS: see_what_works
run: run-tests pathfinder

- name: Run cuda.bindings tests
if: ${{ env.SKIP_CUDA_BINDINGS_TEST == '0' }}
if: ${{ inputs.test-mode == 'standard' && env.SKIP_CUDA_BINDINGS_TEST == '0' }}
env:
CUDA_VER: ${{ matrix.CUDA_VER }}
LOCAL_CTK: ${{ matrix.LOCAL_CTK }}
run: run-tests bindings

- name: Run cuda.bindings benchmarks (smoke test)
if: ${{ env.SKIP_CUDA_BINDINGS_TEST == '0' }}
if: ${{ inputs.test-mode == 'standard' && env.SKIP_CUDA_BINDINGS_TEST == '0' }}
run: |
pip install pyperf
pushd benchmarks/cuda_bindings
python run_pyperf.py --debug-single-value
popd

- name: Run cuda.core tests
if: ${{ inputs.test-mode == 'standard' }}
env:
CUDA_VER: ${{ matrix.CUDA_VER }}
LOCAL_CTK: ${{ matrix.LOCAL_CTK }}
run: run-tests core

- name: Ensure cuda-python installable
if: ${{ inputs.test-mode == 'standard' }}
run: |
if [[ "${{ matrix.LOCAL_CTK }}" == 1 ]]; then
pip install --only-binary=:all: cuda_python*.whl
Expand All @@ -293,6 +321,7 @@ jobs:
fi

- name: Install cuda.pathfinder extra wheels for testing
if: ${{ inputs.test-mode == 'standard' }}
run: |
set -euo pipefail
pushd cuda_pathfinder
Expand All @@ -301,8 +330,48 @@ jobs:
popd

- name: Run cuda.pathfinder tests with all_must_work
if: ${{ inputs.test-mode == 'standard' }}
env:
CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS: all_must_work
CUDA_PATHFINDER_TEST_FIND_NVIDIA_HEADERS_STRICTNESS: all_must_work
CUDA_PATHFINDER_TEST_FIND_NVIDIA_BITCODE_LIB_STRICTNESS: all_must_work
run: run-tests pathfinder

# ── Nightly: install all cuda-python wheels ──
- name: Install cuda-python wheels for nightly testing
if: ${{ inputs.test-mode != 'standard' }}
env:
CUDA_VER: ${{ matrix.CUDA_VER }}
LOCAL_CTK: ${{ matrix.LOCAL_CTK }}
run: run-tests nightly-install

# ── Nightly: PyTorch interop tests ──
- name: Install PyTorch
if: ${{ inputs.test-mode == 'nightly-pytorch' }}
run: |
TORCH_VER="${{ matrix.TORCH_VER }}"
TORCH_CUDA="${{ matrix.TORCH_CUDA }}"
if [[ "$TORCH_VER" == "latest" ]]; then
pip install torch --index-url "https://download.pytorch.org/whl/${TORCH_CUDA}"
else
pip install "torch==${TORCH_VER}" --index-url "https://download.pytorch.org/whl/${TORCH_CUDA}"
fi
python -c "import torch; print(f'PyTorch {torch.__version__}, CUDA {torch.version.cuda}')"

- name: Run PyTorch interop tests
if: ${{ inputs.test-mode == 'nightly-pytorch' }}
run: |
pushd cuda_core
pytest -rxXs -v --durations=0 tests/test_utils.py tests/example_tests/
popd

# ── Nightly: numba-cuda tests ──
- name: Install numba-cuda
if: ${{ inputs.test-mode == 'nightly-numba-cuda' }}
run: |
pip install numba-cuda
python -c "import numba_cuda; print(f'numba-cuda installed')"

- name: Run numba-cuda tests
if: ${{ inputs.test-mode == 'nightly-numba-cuda' }}
run: python -m numba_cuda.numba.cuda.tests
Loading
Loading