Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4aadce2
Add nightly CI pipeline for optional-dependency testing (PyTorch, num…
leofang Apr 29, 2026
6dea4c3
Add concurrency group and fix checks job in ci-nightly.yml
leofang Apr 29, 2026
ac5238c
Temporarily add push trigger to ci-nightly.yml for testing
leofang Apr 29, 2026
cb5aefa
Use shallow clone (fetch-depth: 1) in ci-nightly.yml
leofang Apr 29, 2026
9286598
Fix artifact name mismatch in nightly CI by passing source SHA
leofang Apr 29, 2026
0b7cc50
Fix nightly CI: single pip call, display step, numba cmd, Windows VC++
leofang Apr 29, 2026
edeaa76
Match CUDA_VER to TORCH_CUDA in nightly pytorch matrix
leofang Apr 29, 2026
6f04205
Match CUDA_VER to TORCH_CUDA in nightly pytorch matrix
leofang Apr 29, 2026
da50070
Merge remote-tracking branch 'origin/main' into ci-nightly-optdeps
leofang Apr 30, 2026
cec1d8d
Remove numba-cuda test_linker.py patch (fixed upstream)
leofang Apr 30, 2026
833490e
Merge branch 'ci-nightly-optdeps' of https://github.com/leofang/cuda-…
leofang Apr 30, 2026
2a749d5
Merge branch 'main' into ci-nightly-optdeps
leofang May 1, 2026
bb75dcc
Merge branch 'main' into ci-nightly-optdeps
leofang May 11, 2026
dd909d3
Merge remote-tracking branch 'origin/main' into ci-nightly-optdeps
leofang May 14, 2026
ad2a093
Merge branch 'ci-nightly-optdeps' of https://github.com/leofang/cuda-…
leofang May 14, 2026
cd4277a
Consolidate CI run ID lookups into ci/tools/lookup-run-id
leofang May 14, 2026
901dc09
Remove cuda_pathfinder listing from Windows display step
leofang May 14, 2026
b42a4be
Enforce --tag/--branch mutual exclusivity in lookup-run-id
leofang May 14, 2026
0415bab
Show torch version and numba-cuda in nightly CI job names
leofang May 14, 2026
025a26e
Improve CI job name format with descriptive labels
leofang May 14, 2026
a566781
Handle BINDINGS_SOURCE in nightly install and display steps
leofang May 14, 2026
9fd2f3e
Fix display step guard: skip only for published bindings
leofang May 14, 2026
a232927
Pin nightly pytorch 'latest' to 2.11 (tensor bridge version cap)
leofang May 14, 2026
7d8c286
Use full version 2.11.0 for torch pin
leofang May 14, 2026
2396430
Add arm64 pytorch nightly entries and fix incorrect comment
leofang May 14, 2026
4d1c871
Address rwgk review: cron offset, CUDA_BUILD_VER from source SHA, che…
leofang May 15, 2026
7d36fea
Consolidate core and nightly-* install logic in run-tests
leofang May 15, 2026
f88eddd
revert the test trigger
leofang May 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Consolidate core and nightly-* install logic in run-tests
Merge the duplicated variable setup (TEST_CUDA_MAJOR, CUDA_VER_MINOR,
FREE_THREADING, BINDINGS_ARGS, CORE_WHL) into a shared block for both
core and nightly-* modes.  Use nightly-* glob pattern for validation,
pathfinder guard, and branch dispatch for future extensibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
  • Loading branch information
leofang and claude committed May 15, 2026
commit 7d36fea9dc0360a9d6c62a42953eec8d0f61b45d
154 changes: 66 additions & 88 deletions ci/tools/run-tests
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ if [[ ${#} -ne 1 ]]; then
echo "Error: This script requires exactly 1 argument. You provided ${#}"
exit 1
fi
if [[ "${1}" != "bindings" && "${1}" != "core" && "${1}" != "pathfinder" && "${1}" != "nightly-pytorch" && "${1}" != "nightly-numba-cuda" ]]; then
echo "Error: Invalid test module '${1}'. Must be 'bindings', 'core', 'pathfinder', 'nightly-pytorch', or 'nightly-numba-cuda'"
if [[ "${1}" != "bindings" && "${1}" != "core" && "${1}" != "pathfinder" && "${1}" != nightly-* ]]; then
echo "Error: Invalid test module '${1}'. Must be 'bindings', 'core', 'pathfinder', or 'nightly-*'"
exit 1
fi

Expand All @@ -23,7 +23,7 @@ test_module=${1}
# For standard modes, install pathfinder up front (it is a direct dependency
# of bindings, and a transitive dependency of core). Nightly modes install
# all wheels together in a single pip call further below.
if [[ "${test_module}" != "nightly-pytorch" && "${test_module}" != "nightly-numba-cuda" ]]; then
if [[ "${test_module}" != nightly-* ]]; then
pushd ./cuda_pathfinder
echo "Installing pathfinder wheel"
pip install ./*.whl --group test
Expand Down Expand Up @@ -56,54 +56,8 @@ elif [[ "${test_module}" == "bindings" ]]; then
${SANITIZER_CMD} pytest -rxXs -v --durations=0 --randomly-dont-reorganize tests/cython
fi
popd
elif [[ "${test_module}" == "core" ]]; then
# Install cuda.bindings for core tests based on BINDINGS_SOURCE.
if [[ "${BINDINGS_SOURCE}" == "published" ]]; then
echo "Installing published cuda-bindings==${TEST_CUDA_MAJOR}.${TEST_CUDA_MINOR}.* from PyPI"
pip install "cuda-bindings==${TEST_CUDA_MAJOR}.${TEST_CUDA_MINOR}.*"
elif [[ "${BINDINGS_SOURCE}" == "backport" || "${BINDINGS_SOURCE}" == "main" ]]; then
echo "Installing bindings wheel (source: ${BINDINGS_SOURCE})"
if [[ "${LOCAL_CTK}" == 1 ]]; then
pip install "${CUDA_BINDINGS_ARTIFACTS_DIR}"/*.whl
else
pip install $(ls "${CUDA_BINDINGS_ARTIFACTS_DIR}"/*.whl)[all]
fi
fi
TEST_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${CUDA_VER})"
echo "Installing core wheel"

FREE_THREADING=""
if python -c 'import sys; assert not sys._is_gil_enabled()' 2> /dev/null; then
FREE_THREADING+="-ft"
fi

pushd ./cuda_core
CUDA_VER_MINOR="$(cut -d '.' -f 1-2 <<< "${CUDA_VER}")"
# Start from the built wheel path, then add the published cuda.bindings extra
# when this job is resolving against wheel-installed CTK packages.
WHL_EXTRA=("${CUDA_CORE_ARTIFACTS_DIR}"/*.whl)
if [[ "${LOCAL_CTK}" != 1 ]]; then
WHL_EXTRA=("${WHL_EXTRA[0]}[cu${TEST_CUDA_MAJOR}]")
fi
# Constrain cuda-toolkit to the requested CTK version to avoid
# pip pulling in a newer nvidia-cuda-runtime that conflicts with it.
pip install "${WHL_EXTRA[@]}" --group "test-cu${TEST_CUDA_MAJOR}${FREE_THREADING}" "cuda-toolkit==${CUDA_VER_MINOR}.*"
echo "Installed packages before core tests:"
pip list
echo "Running core tests"
${SANITIZER_CMD} pytest -rxXs -v --durations=0 --randomly-dont-reorganize tests/
# Currently our CI always installs the latest bindings (from either major version).
# This is not compatible with the test requirements.
if [[ "${SKIP_CYTHON_TEST}" == 0 ]]; then
${SANITIZER_CMD} pytest -rxXs -v --durations=0 --randomly-dont-reorganize tests/cython
fi
popd
elif [[ "${test_module}" == "nightly-pytorch" || "${test_module}" == "nightly-numba-cuda" ]]; then
# Nightly optional-dependency testing.
# Install ALL wheels (pathfinder + bindings + core) and the optional dep
# in a single pip call so pip resolves version constraints in one shot
# and avoids costly uninstall/reinstall cycles.

elif [[ "${test_module}" == "core" || "${test_module}" == nightly-* ]]; then
# Shared setup for core and nightly modes.
TEST_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${CUDA_VER})"
CUDA_VER_MINOR="$(cut -d '.' -f 1-2 <<< "${CUDA_VER}")"

Expand All @@ -112,12 +66,7 @@ elif [[ "${test_module}" == "nightly-pytorch" || "${test_module}" == "nightly-nu
FREE_THREADING+="-ft"
fi

# Resolve pathfinder wheel to absolute path before pushd.
# CUDA_BINDINGS_ARTIFACTS_DIR and CUDA_CORE_ARTIFACTS_DIR are already
# absolute (set via realpath in env-vars).
PATHFINDER_WHL=($(realpath ./cuda_pathfinder/*.whl))

# Build bindings spec based on BINDINGS_SOURCE (set by env-vars):
# Resolve bindings based on BINDINGS_SOURCE (set by env-vars):
# main/backport → local wheel from artifacts dir
# published → install from PyPI by version
BINDINGS_ARGS=()
Expand All @@ -130,48 +79,77 @@ elif [[ "${test_module}" == "nightly-pytorch" || "${test_module}" == "nightly-nu
fi
fi

# Resolve core wheel, adding the published cuda.bindings extra
# when this job is resolving against wheel-installed CTK packages.
CORE_WHL=("${CUDA_CORE_ARTIFACTS_DIR}"/*.whl)
if [[ "${LOCAL_CTK}" != 1 ]]; then
CORE_WHL=("${CORE_WHL[0]}[cu${TEST_CUDA_MAJOR}]")
fi

if [[ "${test_module}" == nightly-* ]]; then
# Resolve pathfinder wheel to absolute path before pushd.
# CUDA_BINDINGS_ARTIFACTS_DIR and CUDA_CORE_ARTIFACTS_DIR are already
# absolute (set via realpath in env-vars).
PATHFINDER_WHL=($(realpath ./cuda_pathfinder/*.whl))
fi

# pushd so --group reads test dependency groups from cuda_core/pyproject.toml.
# The explicit cuda-toolkit[...]==X.Y.* pin overrides the group's looser ==X.*.
pushd ./cuda_core

PIP_ARGS=(
"${PATHFINDER_WHL[@]}"
"${BINDINGS_ARGS[@]}"
"${CORE_WHL[@]}"
--group "test-cu${TEST_CUDA_MAJOR}${FREE_THREADING}"
)

if [[ "${test_module}" == "nightly-pytorch" ]]; then
# TORCH_VER and TORCH_CUDA must be set by the caller.
# Use cuda-toolkit[cudart] only — torch brings its own nvcc/nvrtc/etc.
# This avoids version conflicts between our nvidia-* pins and torch's.
echo "Installing pathfinder + bindings + core + test deps + PyTorch ${TORCH_VER} (${TORCH_CUDA})"
PIP_ARGS+=("cuda-toolkit[cudart]==${CUDA_VER_MINOR}.*")
if [[ "${TORCH_VER}" == "latest" ]]; then
PIP_ARGS+=(torch)
else
PIP_ARGS+=("torch==${TORCH_VER}")
if [[ "${test_module}" == "core" ]]; then
echo "Installing bindings (source: ${BINDINGS_SOURCE})"
pip install "${BINDINGS_ARGS[@]}"
echo "Installing core wheel"
# Constrain cuda-toolkit to the requested CTK version to avoid
# pip pulling in a newer nvidia-cuda-runtime that conflicts with it.
pip install "${CORE_WHL[@]}" --group "test-cu${TEST_CUDA_MAJOR}${FREE_THREADING}" "cuda-toolkit==${CUDA_VER_MINOR}.*"
echo "Installed packages before core tests:"
pip list
echo "Running core tests"
${SANITIZER_CMD} pytest -rxXs -v --durations=0 --randomly-dont-reorganize tests/
# Currently our CI always installs the latest bindings (from either major version).
# This is not compatible with the test requirements.
if [[ "${SKIP_CYTHON_TEST}" == 0 ]]; then
${SANITIZER_CMD} pytest -rxXs -v --durations=0 --randomly-dont-reorganize tests/cython
fi
PIP_ARGS+=(--extra-index-url "https://download.pytorch.org/whl/${TORCH_CUDA}")
elif [[ "${test_module}" == "nightly-numba-cuda" ]]; then
echo "Installing pathfinder + bindings + core + test deps + numba-cuda"
# numba-cuda's test-cuXX group deps (can't use --group for a wheel install):
PIP_ARGS+=(
"cuda-toolkit[curand,cublas]==${CUDA_VER_MINOR}.*"
"numba-cuda[cu${TEST_CUDA_MAJOR}]"
"cupy-cuda${TEST_CUDA_MAJOR}x"
psutil cffi pytest-xdist pytest-benchmark filecheck ml_dtypes statistics
else
# Nightly optional-dependency testing.
# Install ALL wheels (pathfinder + bindings + core) and the optional dep
# in a single pip call so pip resolves version constraints in one shot.
PIP_ARGS=(
"${PATHFINDER_WHL[@]}"
"${BINDINGS_ARGS[@]}"
"${CORE_WHL[@]}"
--group "test-cu${TEST_CUDA_MAJOR}${FREE_THREADING}"
)

if [[ "${test_module}" == "nightly-pytorch" ]]; then
# TORCH_VER and TORCH_CUDA must be set by the caller.
# Use cuda-toolkit[cudart] only — torch brings its own nvcc/nvrtc/etc.
# This avoids version conflicts between our nvidia-* pins and torch's.
echo "Installing pathfinder + bindings + core + test deps + PyTorch ${TORCH_VER} (${TORCH_CUDA})"
PIP_ARGS+=("cuda-toolkit[cudart]==${CUDA_VER_MINOR}.*")
if [[ "${TORCH_VER}" == "latest" ]]; then
PIP_ARGS+=(torch)
else
PIP_ARGS+=("torch==${TORCH_VER}")
fi
PIP_ARGS+=(--extra-index-url "https://download.pytorch.org/whl/${TORCH_CUDA}")
elif [[ "${test_module}" == "nightly-numba-cuda" ]]; then
echo "Installing pathfinder + bindings + core + test deps + numba-cuda"
# numba-cuda's test-cuXX group deps (can't use --group for a wheel install):
PIP_ARGS+=(
"cuda-toolkit[curand,cublas]==${CUDA_VER_MINOR}.*"
"numba-cuda[cu${TEST_CUDA_MAJOR}]"
"cupy-cuda${TEST_CUDA_MAJOR}x"
psutil cffi pytest-xdist pytest-benchmark filecheck ml_dtypes statistics
)
fi

pip install "${PIP_ARGS[@]}"
echo "Nightly install complete — installed packages:"
pip list
fi

pip install "${PIP_ARGS[@]}"
popd

echo "Nightly install complete — installed packages:"
pip list
fi
Loading