Skip to content

Commit d21a218

Browse files
authored
CI: Integrate cuda-pathfinder into the shared release workflow (#3046)
* CI: Integrate cuda-pathfinder into the shared release workflow Summary Refs #3012. Integrate Pathfinder’s automated release process into release.yml, sharing documentation and artifact-upload jobs while preserving the release process for other products. - Preserve Pathfinder’s release-note checks, successful tag-triggered CI requirement, draft metadata, checkout-free publishing, and installation verification on TestPyPI and PyPI. - Require successful documentation before uploading assets or publishing packages. Finalize the Pathfinder GitHub release only after successful uploads and PyPI verification, without marking it as latest. - Reuse release-upload.yml for source archives, checksums, and validated wheels. Load Pathfinder validation tools from the workflow revision so older release tags remain supported. - Reduce duplication with YAML anchors for publishing and verification steps. - Preserve Pathfinder release serialization and default-branch enforcement for full releases, while supporting the shared dry-run mode. - Keep other products’ publishing behavior, upload concurrency, and manual GitHub release finalization unchanged. Rollout The existing release-cuda-pathfinder.yml remains unchanged during migration. Pathfinder’s trusted publishers on both PyPI and TestPyPI must authorize release.yml before its first full release. Retire the legacy workflow and publisher registrations afterward. Validation Workflow linting passes. Live workflow-dispatch and trusted-publishing validation remain outstanding. * Implement coderabbit suggestions * Remove cuda-pathfinder specific ordering, but leave cuda-pathfinder specific work, except for verify test/pypi, which is removed. * Fix refs * Remove unnecessary deduplication * Fix coderabbit suggestions * Fix whitespace * Coderabbit commets * Address coderabbit comments * Remove unnecessary additional step * Remove unused --git-ref, and drive-by changes. * Address codex comments. Add a concurrency group per component. Reinstate testpypi and pypi verification steps.
1 parent 17de5e1 commit d21a218

5 files changed

Lines changed: 317 additions & 25 deletions

File tree

‎.github/RELEASE-core.md‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,42 @@
66
This document covers the `cuda.core` release process. For other packages:
77
`cuda-bindings` and `cuda-python` involve a private repository and are not
88
documented here; `cuda-pathfinder` is largely automated by the
9-
[release-cuda-pathfinder.yml](workflows/release-cuda-pathfinder.yml)
9+
[CI: Release](workflows/release.yml)
1010
workflow.
1111

12+
To release `cuda-pathfinder`, dispatch **CI: Release** with
13+
`component` set to `cuda-pathfinder`, `release-action` set
14+
to `full-release`, and `git-tag` set to the existing
15+
`cuda-pathfinder-v<VERSION>` tag. The workflow finds the successful
16+
tag-triggered CI run, checks release notes, creates or reuses a draft,
17+
builds and deploys versioned docs, and uploads the source archive, checksum,
18+
and wheels. It publishes to TestPyPI, verifies installation and the imported
19+
version, then publishes to PyPI and verifies again before publishing the
20+
GitHub Release without marking it as latest. A failed
21+
docs build prevents asset uploads, package publication, and finalization.
22+
23+
The release-note checker runs from `git-tag`, not the workflow revision.
24+
For Pathfinder, that tag must contain a version
25+
of `ci/tools/check_release_notes.py` that writes `version=<VERSION>` to
26+
`GITHUB_OUTPUT`. The release-notes job rejects an empty version before
27+
documentation, asset uploads, or package publication can start. Pathfinder
28+
package publication also requires successful draft metadata preparation.
29+
Use the retained legacy Pathfinder workflow for older tags without version
30+
output support during the transition; do not move existing tags to add the
31+
updated checker.
32+
33+
The default `dry-run` action validates preparation, builds docs, and
34+
collects release artifacts without creating a GitHub Release or publishing
35+
packages. It can be dispatched from a development branch; use
36+
`dry-run-docs-branch` only when a preview deployment is needed.
37+
38+
Before the first Pathfinder release through this workflow, its trusted
39+
publishers on **both PyPI and TestPyPI** must authorize `release.yml`.
40+
Until that migration is complete, `release-cuda-pathfinder.yml` remains
41+
available. Keep its existing publisher
42+
registrations during the transition; retire the old workflow and its
43+
registrations after the consolidated workflow is authorized on both indexes.
44+
1245
Each section below provides detailed guidance for a step in the
1346
[Release Checklist](ISSUE_TEMPLATE/release_checklist.yml). To start a
1447
release, create a new issue from that template and work through it item by

‎.github/workflows/release-upload.yml‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,9 @@ on:
2525
default: false
2626

2727
concurrency:
28-
# Concurrency group that uses the workflow name and PR number if available
29-
# or commit SHA as a fallback. If a new build is triggered under that
30-
# concurrency group while a previous build is running it will be canceled.
31-
# Repeated pushes to a PR will cancel all previous builds, while multiple
32-
# merges to main will not cancel.
33-
group: ${{ github.workflow }}-${{ github.ref_name || github.sha }}
34-
cancel-in-progress: true
28+
# Keep upload locks component-specific and separate from the caller's workflow lock.
29+
group: release-upload-${{ inputs.component }}
30+
cancel-in-progress: false
3531

3632
permissions:
3733
contents: write

‎.github/workflows/release.yml‎

Lines changed: 191 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@ name: "CI: Release"
88
# for the given tag, or uses an existing draft, then publishes the selected wheels
99
# to TestPyPI followed by PyPI. The dry-run mode validates the release path
1010
# without publishing to external release surfaces.
11+
# Pathfinder additionally prepares metadata, verifies both indexes, and publishes its draft.
12+
# Before retiring the dedicated Pathfinder workflow, authorize release.yml in
13+
# its PyPI and TestPyPI trusted publishers; see .github/RELEASE-core.md.
1114
#
1215
# Maintenance note: non-trivial changes to this workflow should be validated
1316
# with dry-run workflow_dispatch runs before merging. Suggested focused matrix:
@@ -29,6 +32,7 @@ name: "CI: Release"
2932
# Leave run-id blank so determine-run-id is exercised. For exhaustive coverage,
3033
# add a mainline cuda-python dry-run when changes could affect metapackage
3134
# artifact validation, docs routing, or component-specific release behavior.
35+
# Also exercise cuda-core and cuda-pathfinder, including a failed docs build.
3236

3337
on:
3438
workflow_dispatch:
@@ -70,6 +74,11 @@ on:
7074
type: string
7175
default: ""
7276

77+
concurrency:
78+
# Serialize releases per component without canceling an active run.
79+
group: release-${{ inputs.component }}
80+
cancel-in-progress: false
81+
7382
defaults:
7483
run:
7584
shell: bash --noprofile --norc -xeuo pipefail {0}
@@ -92,13 +101,16 @@ jobs:
92101
id: lookup-run-id
93102
env:
94103
GH_TOKEN: ${{ github.token }}
104+
RELEASE_GIT_TAG: ${{ inputs.git-tag }}
105+
PROVIDED_RUN_ID: ${{ inputs.run-id }}
106+
RELEASE_REPOSITORY: ${{ github.repository }}
95107
run: |
96-
if [[ -n "${{ inputs.run-id }}" ]]; then
97-
echo "Using provided run ID: ${{ inputs.run-id }}"
98-
RUN_ID="${{ inputs.run-id }}"
108+
if [[ -n "$PROVIDED_RUN_ID" ]]; then
109+
echo "Using provided run ID: $PROVIDED_RUN_ID"
110+
RUN_ID="$PROVIDED_RUN_ID"
99111
else
100-
echo "Auto-detecting successful tag-triggered run ID for tag: ${{ inputs.git-tag }}"
101-
RUN_ID=$(./ci/tools/lookup-run-id "${{ inputs.git-tag }}" "${{ github.repository }}")
112+
echo "Auto-detecting successful tag-triggered run ID for tag: $RELEASE_GIT_TAG"
113+
RUN_ID=$(./ci/tools/lookup-run-id "$RELEASE_GIT_TAG" "$RELEASE_REPOSITORY")
102114
echo "Auto-detected run ID: $RUN_ID"
103115
fi
104116
echo "run-id=$RUN_ID" >> "$GITHUB_OUTPUT"
@@ -158,6 +170,8 @@ jobs:
158170
check-release-notes:
159171
if: ${{ github.repository_owner == 'nvidia' }}
160172
runs-on: ubuntu-latest
173+
outputs:
174+
version: ${{ steps.release-notes.outputs.version }}
161175
steps:
162176
- name: Checkout Source
163177
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -170,11 +184,74 @@ jobs:
170184
python-version: "3.12"
171185

172186
- name: Check versioned release notes exist
187+
id: release-notes
188+
env:
189+
RELEASE_GIT_TAG: ${{ inputs.git-tag }}
190+
RELEASE_COMPONENT: ${{ inputs.component }}
191+
BACKPORT_GIT_TAG: ${{ inputs.backport-git-tag }}
173192
run: |
174193
python ci/tools/check_release_notes.py \
175-
--git-tag "${{ inputs.git-tag }}" \
176-
--component "${{ inputs.component }}" \
177-
--backport-git-tag "${{ inputs.backport-git-tag }}"
194+
--git-tag "$RELEASE_GIT_TAG" \
195+
--component "$RELEASE_COMPONENT" \
196+
--backport-git-tag "$BACKPORT_GIT_TAG"
197+
198+
- name: Require Pathfinder version output
199+
if: ${{ inputs.component == 'cuda-pathfinder' }}
200+
env:
201+
VERSION: ${{ steps.release-notes.outputs.version }}
202+
run: |
203+
[[ -n "$VERSION" ]] || {
204+
echo "::error::The tagged Pathfinder release-note checker must emit a non-empty version output"
205+
exit 1
206+
}
207+
208+
ensure-pathfinder-draft:
209+
name: Prepare Pathfinder release draft
210+
if: ${{ github.repository_owner == 'nvidia' && inputs.component == 'cuda-pathfinder' && inputs.release-action == 'full-release' }}
211+
needs: [check-tag, check-release-notes]
212+
runs-on: ubuntu-latest
213+
permissions:
214+
contents: write
215+
env:
216+
TAG: ${{ inputs.git-tag }}
217+
VERSION: ${{ needs.check-release-notes.outputs.version }}
218+
GH_TOKEN: ${{ github.token }}
219+
steps:
220+
- name: Set metadata for a Pathfinder draft
221+
run: |
222+
[[ -n "$VERSION" ]] || { echo "::error::Empty version output"; exit 1; }
223+
# Recognize the shared placeholder across retries without overwriting curated drafts.
224+
is_placeholder=$(gh release view "$TAG" --repo "${{ github.repository }}" --json isDraft,name,body \
225+
| jq -r --arg placeholder "Release ${TAG}" '
226+
if .isDraft != true then error("Release is not a draft")
227+
else .name == $placeholder and .body == $placeholder end
228+
')
229+
if [[ "$is_placeholder" != "true" ]]; then
230+
echo "Reusing draft release ${TAG}"
231+
exit 0
232+
fi
233+
cat > "${RUNNER_TEMP}/release-body.md" <<BODY
234+
## Release notes
235+
236+
- https://nvidia.github.io/cuda-python/cuda-pathfinder/latest/release/${VERSION}-notes.html
237+
238+
## Documentation
239+
240+
- https://nvidia.github.io/cuda-python/cuda-pathfinder/${VERSION}/
241+
242+
## PyPI
243+
244+
- https://pypi.org/project/cuda-pathfinder/${VERSION}/
245+
246+
## Conda
247+
248+
- https://anaconda.org/conda-forge/cuda-pathfinder/files?version=${VERSION}
249+
- \`conda install conda-forge::cuda-pathfinder=${VERSION}\`
250+
BODY
251+
gh release edit "$TAG" --repo "${{ github.repository }}" \
252+
--latest=false \
253+
--title "cuda-pathfinder v${VERSION}" \
254+
--notes-file "${RUNNER_TEMP}/release-body.md"
178255
179256
doc:
180257
name: Build release docs
@@ -217,13 +294,20 @@ jobs:
217294

218295
publish-testpypi:
219296
name: Publish wheels to TestPyPI
220-
if: ${{ inputs.release-action == 'full-release' }}
297+
if: >-
298+
${{ !cancelled() && inputs.release-action == 'full-release' &&
299+
needs.check-tag.result == 'success' &&
300+
needs.check-release-notes.result == 'success' &&
301+
needs.determine-run-id.result == 'success' &&
302+
needs.doc.result == 'success' &&
303+
(inputs.component != 'cuda-pathfinder' || needs.ensure-pathfinder-draft.result == 'success') }}
221304
runs-on: ubuntu-latest
222305
needs:
223306
- check-tag
224307
- check-release-notes
225308
- determine-run-id
226309
- doc
310+
- ensure-pathfinder-draft
227311
environment:
228312
name: testpypi
229313
url: https://test.pypi.org/p/${{ inputs.component }}/
@@ -248,13 +332,59 @@ jobs:
248332
with:
249333
repository-url: https://test.pypi.org/legacy/
250334

335+
verify-testpypi:
336+
name: Verify Pathfinder on TestPyPI
337+
if: ${{ inputs.component == 'cuda-pathfinder' && inputs.release-action == 'full-release' }}
338+
needs: [check-release-notes, publish-testpypi]
339+
runs-on: ubuntu-latest
340+
permissions: {}
341+
env:
342+
VERSION: ${{ needs.check-release-notes.outputs.version }}
343+
INDEX_URL: https://test.pypi.org/simple/
344+
EXTRA_INDEX_URL: https://pypi.org/simple/
345+
INDEX_NAME: TestPyPI
346+
steps: &verify-pathfinder-steps
347+
- name: Install Pathfinder from index and verify
348+
run: |
349+
[[ -n "$VERSION" ]] || { echo "::error::Empty version output"; exit 1; }
350+
python3 -m venv "${RUNNER_TEMP}/verify"
351+
source "${RUNNER_TEMP}/verify/bin/activate"
352+
index_args=()
353+
if [[ -n "$INDEX_URL" ]]; then
354+
index_args+=( --index-url "$INDEX_URL" )
355+
fi
356+
if [[ -n "$EXTRA_INDEX_URL" ]]; then
357+
index_args+=( --extra-index-url "$EXTRA_INDEX_URL" )
358+
fi
359+
max_attempts=6
360+
retry_seconds=30
361+
for ((attempt=1; attempt<=max_attempts; attempt++)); do
362+
if pip install "${index_args[@]}" "cuda-pathfinder==${VERSION}"; then
363+
break
364+
fi
365+
if (( attempt == max_attempts )); then
366+
echo "::error::Failed to install cuda-pathfinder==${VERSION} from ${INDEX_NAME} after ${max_attempts} attempts"
367+
exit 1
368+
fi
369+
sleep "$retry_seconds"
370+
done
371+
installed=$(python -c "from cuda.pathfinder import __version__; print(__version__)")
372+
if [[ "$installed" != "$VERSION" ]]; then
373+
echo "::error::Version mismatch: expected ${VERSION}, got ${installed}"
374+
exit 1
375+
fi
376+
251377
publish-pypi:
252378
name: Publish wheels to PyPI
253-
if: ${{ inputs.release-action == 'full-release' }}
379+
if: >-
380+
${{ !cancelled() && inputs.release-action == 'full-release' &&
381+
needs.determine-run-id.result == 'success' && needs.publish-testpypi.result == 'success' &&
382+
(inputs.component != 'cuda-pathfinder' || needs.verify-testpypi.result == 'success') }}
254383
runs-on: ubuntu-latest
255384
needs:
256385
- determine-run-id
257386
- publish-testpypi
387+
- verify-testpypi
258388
environment:
259389
name: pypi
260390
url: https://pypi.org/p/${{ inputs.component }}/
@@ -277,4 +407,54 @@ jobs:
277407
- name: Publish package distributions to PyPI
278408
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
279409

280-
# TODO: add another job to make the release leave the draft state?
410+
verify-pypi:
411+
name: Verify Pathfinder on PyPI
412+
if: ${{ inputs.component == 'cuda-pathfinder' && inputs.release-action == 'full-release' }}
413+
needs: [check-release-notes, publish-pypi]
414+
runs-on: ubuntu-latest
415+
permissions: {}
416+
env:
417+
VERSION: ${{ needs.check-release-notes.outputs.version }}
418+
INDEX_URL: ""
419+
EXTRA_INDEX_URL: ""
420+
INDEX_NAME: PyPI
421+
steps: *verify-pathfinder-steps
422+
423+
finalize-pathfinder:
424+
name: Publish Pathfinder GitHub release
425+
if: ${{ inputs.component == 'cuda-pathfinder' && inputs.release-action == 'full-release' }}
426+
needs: [doc, check-release-notes, ensure-pathfinder-draft, publish-pypi, verify-pypi, upload-archive]
427+
runs-on: ubuntu-latest
428+
permissions:
429+
contents: write
430+
env:
431+
TAG: ${{ inputs.git-tag }}
432+
VERSION: ${{ needs.check-release-notes.outputs.version }}
433+
steps:
434+
- name: Verify docs URL
435+
run: |
436+
[[ -n "$VERSION" ]] || { echo "::error::Empty version output"; exit 1; }
437+
url="https://nvidia.github.io/cuda-python/cuda-pathfinder/${VERSION}/"
438+
status=$(curl -sL -o /dev/null -w '%{http_code}' "$url")
439+
if [[ "$status" != "200" ]]; then
440+
echo "::warning::Docs URL returned HTTP ${status} -- docs may not be deployed yet"
441+
fi
442+
443+
- name: Verify release is still a draft
444+
env:
445+
GH_TOKEN: ${{ github.token }}
446+
run: |
447+
is_draft=$(gh release view "$TAG" --repo "${{ github.repository }}" --json isDraft --jq '.isDraft')
448+
if [[ "$is_draft" != "true" ]]; then
449+
echo "::error::Release ${TAG} is no longer a draft (was it published manually?)"
450+
exit 1
451+
fi
452+
453+
- name: Publish release
454+
env:
455+
GH_TOKEN: ${{ github.token }}
456+
run: |
457+
gh release edit "$TAG" --repo "${{ github.repository }}" --draft=false --latest=false
458+
docs_url="https://nvidia.github.io/cuda-python/cuda-pathfinder/${VERSION}/"
459+
echo "$docs_url"
460+
printf '%s\n' "$docs_url" >> "$GITHUB_STEP_SUMMARY"

0 commit comments

Comments
 (0)