Skip to content

CI: build cupy-cuda{12,13}x wheels on GH-hosted runners, gated behind /test - #10049

Merged
leofang merged 21 commits into
cupy:mainfrom
leofang:ci/build-stage-shared-infra
Aug 10, 2026
Merged

leofang merged 21 commits into
cupy:mainfrom
leofang:ci/build-stage-shared-infra

Conversation

@leofang

@leofang leofang commented Jun 28, 2026 •

Copy link
Copy Markdown
Member

First step toward #9794 (split build/test) and #9974 (nightly wheels). Adds a CPU-only build pipeline on free ubuntu-22.04 / windows-2022 runners that produces release-ready cupy-cuda{12,13}x wheels — also reusable as nightlies (e.g. push to the scientific-python nightly channel) via the same artifacts.

Scope: linux-64 + win-64 × CTK {12.x, 13.x} × Python {3.10–3.14, 3.14t}.

The design of the new CI infrastructure and the majority of new code came straight from NVIDIA/cuda-python.

Build acceleration via sccache

NVCC is wrapped through sccache (rapidsai fork, for nvcc/cicc/ptxas caching) backed by the GHA Cache:

  • warm cache: the full 24-wheel matrix completes in ~14 min wall clock, 5–8 min per job (warm run)
  • cold cache: 20–57 min per job uncached (first populated run — late-starting jobs there already ran faster as blobs landed mid-run).

"Release-ready" wheel builds

Release-grade wheels also need the packaging machinery that today lives in cupy-release-tools; rather than pulling that repo into CI, a minimal subset is vendored in-tree.1

  • ci/tools/wheel_configs.py — subset of cupy-release-tools' dist_config.py: per-CTK package names (cupy-cuda12x/cupy-cuda13x), the preload-library matrix (cuTENSOR + NCCL on Linux, cuTENSOR on Windows), and the long_description templates.
  • ci/tools/prepare_wheel_build.py — runs before cibuildwheel (after cupyx/tools/install_library.py stages the preload libraries): renames project.name in pyproject.toml, writes description.rst, generates the _wheel.json preload metadata backing the [ctk] extras, and emits the CUPY_* env vars the build consumes (its docstring has local-reproduction steps).

These stay in sync with cupy-release-tools until that repo is retired in favor of the in-tree definitions (part of the #9974 goal).

/test-driven orchestration as three single-purpose workflows

UX is unchanged: /test <target> comments from an OWNER/MEMBER (and PR merges) drive the pipeline; regular PR updates only run the lightweight pre-flight checks. The model is now split across three workflows, each reacting to one kind of event (details live in each file's header comment):

  1. ci-trigger.yml (issue_comment) — validates the /test request, records it, and bounces a ci:triggered label using a code-less GitHub App token.2
  2. ci.yml (pull_request: labeled; push) — the build leg, running under a real pull_request event: the only remaining way to get per-PR sccache isolation with warm-up from main's cache.3 Wheels are named …-pr<N>-<head sha> (PR) / …-<sha> (push) so the FlexCI-side test stage (CI: wire .pfnci test stage to GHA-built wheels #10058) can reconstruct the names from its own checkout. Push runs also dispatch FlexCI after the builds.
  3. flexci.yml (workflow_run) — the /test dispatch leg: verifies the completed build and the recorded request, then feeds the original payload to the unchanged flexci_dispatcher.py.

The build leg was smoke-tested pre-merge by applying the label manually (run 29797872324: 24 wheels + sentinel, dispatch correctly skipped); the other two legs execute the default branch's workflow copies, so they are only testable post-merge. The workflows must be backported to v14 in lockstep with this PR's merge (the temporary legacy-dispatch job in flexci.yml covers v14 until then).

Note for reviewers

  • pretest.yml's build-cuda cache (one ~385 MB ~/.cache entry per merge SHA) is temporarily disabled so it doesn't crowd the GHA Cache budget the wheel builds rely on; its keying will be reworked in a follow-up PR for Reduce wasteful GHA Cache usage #10059.

Footnotes

  1. Reusing the repo as-is or as a submodule doesn't fit: its build engine bakes the full CTK into per-CTK Docker images, which collides with the manylinux + mini-CTK + cibuildwheel pattern used here; only ~120 lines of packaging config were actually reusable. ↩

  2. Events created with a workflow's own GITHUB_TOKEN never trigger workflows. Repo-side setup (thanks @kmaehashi!): the cupy-ci-trigger App (Pull requests: write, to add/remove the ci:triggered label on PRs) and a ci-trigger environment holding its credentials. ↩

  3. Read-only Actions cache for untrusted triggers (June 2026) — building straight from the comment event would rebuild cold on every /test. ↩

@leofang
leofang force-pushed the ci/build-stage-shared-infra branch 10 times, most recently from dbe774f to cbb60d9 Compare June 29, 2026 00:52
Pure verbatim copy of cuda-python's build-stage CI as the starting
point for CuPy's shared-infra port. SPDX headers stripped (CuPy
convention). CuPy-specific adaptations follow in separate commits so
reviewers can diff against the source.

Workflows:
- .github/workflows/ci.yml          (orchestrator)
- .github/workflows/build-wheel.yml (reusable wheel build)

Composite actions:
- .github/actions/fetch_ctk/         (downloads mini CTK from redist)
- .github/actions/install_unix_deps/ (apt deps on Ubuntu)
- .github/actions/sccache-summary/   (parses sccache stats JSON)

Helpers:
- ci/versions.yml                  (CTK build pins)
- ci/tools/fetch_ctk_redistrib.py  (component filter / relative_path)
- ci/tools/lookup-run-id           (find prior GHA run)
- ci/tools/env-vars                (per-job env var setup)

Source: NVIDIA/cuda-python @ ea0215fd095a6ae86af1b01b2c6c35f725cf4330

Refs: cupy#9794, cupy#9974
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch 2 times, most recently from b872ec2 to 3aedb3c Compare June 29, 2026 01:43
leofang added 5 commits June 29, 2026 01:47
These two scripts have no cuda-python analogue — they exist to bridge
the cuda-python build-stage CI (vendored + adapted in the prior two
commits) with CuPy-specific build conventions inherited from
cupy/cupy-release-tools.

  ci/tools/wheel_configs.py
    Vendored subset of cupy/cupy-release-tools/dist_config.py:
    WHEEL_PACKAGE_NAMES (cupy-cuda{12,13}x), PRELOAD_LIBRARIES per
    platform/CTK, the long-description rST template. Long-term goal
    in cupy#9974 is to retire cupy-release-tools once this pipeline
    reaches parity — this is the first piece pulled in.

  ci/tools/prepare_wheel_build.py
    Glue script called from build-wheel.yml. Rewrites pyproject.toml's
    project.name, writes description.rst, generates _wheel.json via
    cupy/cupyx/tools/_generate_wheel_metadata.py, globs the preload
    directory for include/lib subdirs, and emits CuPy-specific env
    vars (CUPY_INSTALL_NO_RPATH, CUPY_INSTALL_LONG_DESCRIPTION,
    CUPY_INSTALL_WHEEL_METADATA, CUPY_INCLUDE_PATH, CUPY_LIBRARY_PATH,
    CUPY_PACKAGE_NAME) on stdout for $GITHUB_ENV capture. Docstring
    has a one-shell-script local reproduction.

The heavier reusable bits — install_library.py and
_generate_wheel_metadata.py — already live in cupy/cupyx/tools/ and
are invoked directly by build-wheel.yml and prepare_wheel_build.py
respectively, so nothing else needs to be vendored here.

Refs: cupy#9794, cupy#9974
Override fetch_ctk's default component list with the set CuPy actually
needs to build. Adds libcublas/libcufft/libcurand/libcusparse/libcusolver
(CuPy includes headers and links libs from all of these), cuda_nvtx
(for nvtx3/nvToolsExt.h used by the profiler), cuda_cuxxfilt (for
libcufilt.a). Drops libcufile and libcudla (not used by CuPy) plus
nvjpeg/npp/cupti from the fetch_ctk default.

Refs: cupy#9794
Enable git symlinks before checkout, verify LongPathsEnabled, and skip
autopep8/ruff on ci/ (vendored cuda-python helpers don't match CuPy's
79-col line length).

Refs: cupy#9794
Add \$CUDA_PATH/lib64/stubs to LIBRARY_PATH so -lcuda resolves at link
time. Exclude libcuda.so* / nvcuda.dll from auditwheel and delvewheel
repair — the driver is provided by the GPU at runtime and must never
be bundled.

Refs: cupy#9794
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch 6 times, most recently from 740d65f to fbbe1a8 Compare June 29, 2026 20:12
Sets NVCC env var so CuPy's install_build.get_nvcc_path() routes nvcc
invocations through sccache, the same way CC/CXX wrap gcc. Drops the
linux-only guard on the rapidsai/sccache install step and branches on
host-platform for URL triple + binary name; uname -m on Linux so the
same path works for the future linux-aarch64 matrix. Install to a
portable workspace-relative ./sccache_bin/, prepended to $GITHUB_PATH
so bare-name 'sccache' / 'nvcc' resolve on Windows (fetch_ctk already
puts $CUDA_PATH/bin on $GITHUB_PATH).

CIBW_ENVIRONMENT (platform-agnostic) holds the shared GHA-cache /
sccache wiring; CIBW_ENVIRONMENT_{LINUX,WINDOWS} hold only the
platform-specific paths (/host/+/project/ inside the manylinux
container vs cygpath/native on Windows).

cl.exe wrapping (cuda-python#1185-style shim) is NOT done; this covers
the dominant nvcc cost. Follow up only if Windows wins are modest.

Refs: cupy#9794
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch 5 times, most recently from 53dee4e to f269e78 Compare June 30, 2026 02:12
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch from 0da5ceb to 1ada881 Compare July 19, 2026 04:05
@leofang leofang removed the ci:triggered Triggers the CI build pipeline (bounced automatically by /test) label Jul 19, 2026
Two independent cleanups squashed (both previously reviewed):

- ci/tools/lookup-run-id was vendored verbatim from cuda-python for the
  planned nightly workflow (cupy#9974), which never landed in this PR;
  nothing on this branch calls it. Both artifact-fetch sites resolve
  artifacts by name via the Actions artifacts API. The nightly
  follow-up needs an adapted lookup anyway (this copy's --tag mode
  expects push-event CI runs of a kind this workflow does not produce).

- cibuildwheel evaluates CIBW_ENVIRONMENT with shell-style parsing on
  all platforms, so the unquoted Windows path lost its backslashes and
  the sccache disk level landed at C:Usersrunneradmin.cachesccache
  (visible in run logs). Single-quote the value so the path survives.
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch from 9a98b59 to c8d76f2 Compare July 20, 2026 04:33
…kflows

Replaces the previous single-workflow orchestration (reviewed over five
design rounds; see PR discussion) with three workflows, each with a
one-sentence story:

  ci-trigger.yml  Turns /test comments from an OWNER/MEMBER into
                  ci:triggered label events, and keeps the label honest
                  on pushes/reopens.
  ci.yml          Builds wheels when a PR is vouched (labeled) or when a
                  merge lands (push); the push leg also dispatches
                  FlexCI in-run.
  flexci.yml      After a successful /test build run, dispatches FlexCI
                  with the recorded request.

Design bar: as safe as the old flexci.yml with the same UX -- not
safer -- except that the new wheel build stage's shared compile cache
must not be poisonable by vouched PR code, which is the single
requirement forcing /test builds into PR-scoped cache context.

/test flow: the trigger leg validates the comment (PR open, OWNER or
MEMBER, explicit target), records the raw event payload as a
dispatch-request-<pr>-<head_sha> artifact, and bounces the ci:triggered
label with a GitHub App token (GITHUB_TOKEN label events do not trigger
workflows). The labeled build runs in the PR's cache scope (writes
isolated per PR, restores fall back to base/default scope -- since
GitHub's 2026-06-26 change, untrusted default-scope contexts are cache
read-only anyway) from the event-pinned merge commit, tags its wheel
artifacts with the PR number (same-head PRs against different bases
never share wheels), and uploads a build sentinel only when every build
succeeded. flexci.yml reacts to the run's completion: sentinel =
liveness (no-op label runs can never dispatch); PR identity comes only
from trusted API metadata (pulls?head=<head repo>:<branch>); per
candidate PR it requires a request recorded by a default-branch run of
this repository (fork runs cannot forge that provenance), and verifies
the PR still points at the tested head and base before feeding the
recorded payload to flexci_dispatcher.py unchanged.

Post-merge: push events (main, v*, hotfix-*) are trusted, carry
secrets, may write this branch's cache scope, and are the dispatcher's
native input -- so build and dispatch live in one run with plain needs:
ordering, no artifacts, no relay. The guard maps the pushed commit to
its merged PR to honor skip-ci, and skips branch deletions. Push runs
have no shared concurrency group: a burst of merges must not cancel
intermediate runs, each of which carries its own dispatch.

A push to a PR (synchronize) cancels any in-flight labeled build via
the shared ci-build-<PR> concurrency group and clears the stale label;
adding an existing label fires no event, so the bounce removes it
first. All artifact uploads set overwrite so "re-run all jobs" stays
viable across immutable v4+ artifact attempts.

TEMPORARY: flexci.yml carries a legacy pull_request_target(closed)
dispatch job for bases not yet backported the new push workflow
(allowlist, currently v14); each allowlist entry is removed by the same
PR that backports the workflows to that base.

The reusable build workflow no longer receives secrets: it needs none,
and labeled runs of same-repo PRs would otherwise expose repository
secrets to PR code.

Requires (already provisioned or requested): the ci:triggered label;
GitHub App cupy-ci-trigger (Issues:R/W, webhook off) with repository
variable CI_TRIGGER_APP_ID and secret CI_TRIGGER_APP_PRIVATE_KEY;
secrets.FLEXCI_WEBHOOK_SECRET; CUPY_CI_GITHUB_TOKEN on the FlexCI side.
The .pfnci wheel fetch (PR cupy#10058) must pin artifact producers and use
the PR-tagged names; the v14 backport of these workflows lands in
lockstep with this PR's merge.
@leofang
leofang force-pushed the ci/build-stage-shared-infra branch from c8d76f2 to e7f9a75 Compare July 20, 2026 04:55
The App credentials (CI_TRIGGER_APP_ID variable and
CI_TRIGGER_APP_PRIVATE_KEY secret) are provisioned in a deployment
environment rather than at repository level, so only jobs declaring
`environment: ci-trigger` can read them. The environment's branch
policy allows main only -- the ref issue_comment runs execute on --
which additionally shuts out every other context. Each /test records
a deployment to the environment as a side effect (a useful audit
trail).
@leofang leofang added the ci:triggered Triggers the CI build pipeline (bounced automatically by /test) label Jul 21, 2026
Instead of hard-coding which release bases still lack the new push
workflow (and paying for it with a coordinated main-side cleanup PR
plus a double-dispatch window after each backport), probe the merge
commit's own tree: the push run for a merge uses exactly those files,
so "new ci.yml present" means that run dispatches and the legacy job
stays quiet. The handoff is atomic -- the very backport merge that
brings the workflow to a base is already dispatched by its own push
run -- so double dispatch and missed dispatch are impossible by
construction, and the job self-retires (delete whenever, no timing
constraint).

The probe only applies to bases the push branch filter can match
(v*/hotfix-*, never crossing '/' -- Actions glob semantics, unlike
bash); merges into any other base always dispatch legacy-style, which
also restores the old any-base behavior for workflow-less branches the
v14 allowlist never covered.

Reviewed by Codex over three rounds (round 1: push-filter gating;
round 2: '/' glob semantics; round 3: no material findings).
@leofang
leofang marked this pull request as ready for review July 22, 2026 04:21
actions/checkout v7 (backported to every maintained line on 2026-07-20,
which is what broke the old flexci.yml on main via the floating v4 tag)
refuses to check out fork-PR code -- head SHA, refs/pull/*, and
merge_commit_sha included -- from pull_request_target and workflow_run
workflows unless the step sets allow-unsafe-pr-checkout: true.

This file's pinned v6.0.3 predates the guard, so nothing is broken
today, but the pin will not stay frozen forever (renovate is active).
Adopt v7.0.1 in this file now and make the two affected checkouts'
security posture explicit:

- the workflow_run untrusted checkout is data only (the dispatcher
  executes from the trusted checkout and merely json-loads
  config.tags.json from it);
- the legacy-dispatch checkout only ever sees merged PRs, i.e.
  reviewed code already landed on the base branch.

The trusted default self-checkout stays unflagged on purpose: the
guard exempts it, and keeping the flag off means any future
explicit-ref checkout added here gets refused until reviewed.

Other workflows keep their current pin: their events (pull_request,
push) are outside the guard's scope on any checkout version.

Reviewed by Codex (plan round: agreed, checkout-site inventory
independently re-derived; diff round: no material findings, pin SHA
and v7.0.1 delta independently verified).
leofang added a commit to leofang/cupy that referenced this pull request Jul 23, 2026
The FlexCI test pipelines (.pfnci/{linux,windows}) now pip-install the
wheel artifact produced by the GHA build stage (added in cupy#10049) instead
of source-building CuPy in every test job.

Wheel fetch (linux/tests/actions/build.sh + windows/test.ps1):

- Derives the artifact suffix ci-guard.sh emits: `pr<N>-<head sha>` for
  /test-driven pull_request builds, `<sha>` post-merge. On Linux,
  PULL_REQUEST is forwarded by run.sh from FlexCI; on Windows,
  IsPullRequestTest + GetPullRequestNumber parse FLEXCI_BRANCH
  (refs/pull/<N>/head).

- Producer pin: the wheel must come from a successful ci.yml run
  (workflow file by path) at the tested SHA under the expected event
  (pull_request for /test, push post-merge), via GET
  /actions/workflows/ci.yml/runs. `path` and `event` are API metadata
  the artifact producer can't forge -- a sibling workflow that shares
  the tested head SHA cannot satisfy this filter.

- Download by immutable artifact ID via GET /actions/artifacts/<id>/zip,
  not by name, so re-run-attempt artifacts can't shadow the intended
  one. Windows routes the binary download through cmd.exe /c to bypass
  PowerShell's UTF-16 stdout redirect (which would corrupt the zip).

- FlexCI's checkout may be the tested commit directly, or a merge of
  that commit into its base -- both are tried (HEAD, then HEAD^2).

Token + tools plumbing:

- linux/run.sh forwards CUPY_CI_GITHUB_TOKEN into the test container.
  Name-only --env + ${VAR+x} set-check keeps the token value out of
  the `set -x` trace.

- generate.py + Linux test Dockerfiles install a pinned `gh` CLI
  (v2.95.0).

CUPY_CI_GITHUB_TOKEN is provisioned FlexCI-side as a fine-grained PAT
(Actions:Read on cupy/cupy). FlexCI targets whose (CUDA, Python) tuple
is outside the wheel-build matrix fail at the fetch step by design,
until the matrix expands.
seberg
seberg previously approved these changes Jul 23, 2026

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, my bot thinks this is all good, the nit's mostly seem not worth the trouble. (I made two comments, though)

It looks like a nice improvement of the CI workflow overall, but I can't say I have quite a full mental model yet, so have to rely largely on the agent to spot any issues.
But, either way, I think we probably should move forward here, although I am not sure if Kenichi-san wants to take a look.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved
include_dirs.append(version_dir / "include")
for lib_subdir in ("lib", "lib64"):
if (version_dir / lib_subdir).is_dir():
library_dirs.append(version_dir / lib_subdir)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bot was oddly persistent that skipping libs shouldn't be needed because they are always there.
To me it seems still reasonable to just skip over though (and rely on later tests to fail if cutensor should be there but isn't).

leofang added a commit to leofang/cupy that referenced this pull request Jul 23, 2026
The FlexCI test pipelines (.pfnci/{linux,windows}) now pip-install the
wheel artifact produced by the GHA build stage (added in cupy#10049) instead
of source-building CuPy in every test job.

Wheel fetch (linux/tests/actions/build.sh + windows/test.ps1):

- Derives the artifact suffix ci-guard.sh emits: `pr<N>-<head sha>` for
  /test-driven pull_request builds, `<sha>` post-merge. On Linux,
  PULL_REQUEST is forwarded by run.sh from FlexCI; on Windows,
  IsPullRequestTest + GetPullRequestNumber parse FLEXCI_BRANCH
  (refs/pull/<N>/head).

- Producer pin: the wheel must come from a successful ci.yml run
  (workflow file by path) at the tested SHA under the expected event
  (pull_request for /test, push post-merge), via GET
  /actions/workflows/ci.yml/runs. `path` and `event` are API metadata
  the artifact producer can't forge -- a sibling workflow that shares
  the tested head SHA cannot satisfy this filter.

- Download by immutable artifact ID via GET /actions/artifacts/<id>/zip,
  not by name, so re-run-attempt artifacts can't shadow the intended
  one. Windows routes the binary download through cmd.exe /c to bypass
  PowerShell's UTF-16 stdout redirect (which would corrupt the zip).

- FlexCI's checkout may be the tested commit directly, or a merge of
  that commit into its base -- both are tried (HEAD, then HEAD^2).

Token + tools plumbing:

- linux/run.sh forwards CUPY_CI_GITHUB_TOKEN into the test container.
  Name-only --env + ${VAR+x} set-check keeps the token value out of
  the `set -x` trace.

- generate.py + Linux test Dockerfiles install a pinned `gh` CLI
  (v2.95.0).

CUPY_CI_GITHUB_TOKEN is provisioned FlexCI-side as a fine-grained PAT
(Actions:Read on cupy/cupy). FlexCI targets whose (CUDA, Python) tuple
is outside the wheel-build matrix fail at the fetch step by design,
until the matrix expands.
leofang added a commit to leofang/cupy that referenced this pull request Jul 23, 2026
The FlexCI test pipelines (.pfnci/{linux,windows}) now pip-install the
wheel artifact produced by the GHA build stage (added in cupy#10049) instead
of source-building CuPy in every test job.

Wheel fetch (linux/tests/actions/build.sh + windows/test.ps1):

- Derives the artifact suffix ci-guard.sh emits: `pr<N>-<head sha>` for
  /test-driven pull_request builds, `<sha>` post-merge. On Linux,
  PULL_REQUEST is forwarded by run.sh from FlexCI; on Windows,
  IsPullRequestTest + GetPullRequestNumber parse FLEXCI_BRANCH
  (refs/pull/<N>/head).

- Producer pin: the wheel must come from a successful ci.yml run
  (workflow file by path) at the tested SHA under the expected event
  (pull_request for /test, push post-merge), via GET
  /actions/workflows/ci.yml/runs. `path` and `event` are API metadata
  the artifact producer can't forge -- a sibling workflow that shares
  the tested head SHA cannot satisfy this filter.

- Download by immutable artifact ID via GET /actions/artifacts/<id>/zip,
  not by name, so re-run-attempt artifacts can't shadow the intended
  one. Windows routes the binary download through cmd.exe /c to bypass
  PowerShell's UTF-16 stdout redirect (which would corrupt the zip).

- FlexCI's checkout may be the tested commit directly, or a merge of
  that commit into its base -- both are tried (HEAD, then HEAD^2).

Token + tools plumbing:

- linux/run.sh forwards CUPY_CI_GITHUB_TOKEN into the test container.
  Name-only --env + ${VAR+x} set-check keeps the token value out of
  the `set -x` trace.

- generate.py + Linux test Dockerfiles install a pinned `gh` CLI
  (v2.95.0).

CUPY_CI_GITHUB_TOKEN is provisioned FlexCI-side as a fine-grained PAT
(Actions:Read on cupy/cupy). FlexCI targets whose (CUDA, Python) tuple
is outside the wheel-build matrix fail at the fetch step by design,
until the matrix expands.
The three CI workflows declared their write scopes (pull-requests,
statuses, issues) at the workflow level, so every job -- including the
untrusted-PR-code build jobs and the reusable-workflow callers --
received them even though only the FlexCI-dispatch and label-hygiene
jobs write anything. Move each scope down to the job that uses it and
default every workflow to contents:read, so a token leak from any build
or PR-controlled job cannot write.

- ci.yml: default contents:read; guard +pull-requests:read (ci-guard.sh
  reads commits/<sha>/pulls on the push leg); dispatch-flexci
  +pull-requests:write, statuses:write (flexci_dispatcher.py). ci-vars
  and the two reusable build callers pin contents:read explicitly; the
  sentinel and status-check jobs inherit the read-only default.
- ci-trigger.yml: default contents:read; test-request pull-requests:read
  (the ci:triggered label bounce uses a separate GitHub App token, not
  GITHUB_TOKEN); hygiene issues:write.
- flexci.yml: default contents:read; dispatch actions:read +
  pull-requests:write + statuses:write; legacy-dispatch pull-requests:write
  + statuses:write (no actions:read -- it reads no run artifacts).
- build-wheel.yml is unchanged: it already declares its own contents:read
  ceiling (a reusable workflow does not inherit the caller's permissions,
  so each file is self-contained).

Addresses seberg's review: the guard and ci-vars per-job permission
suggestions are applied verbatim.

Reviewed by Codex (working-tree diff: the token scopes needed by each
affected job are preserved, no regression).

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought I had put the green checkmark. Anyway, I think this is good to try (but maybe the timing now means we should just wait)...

leofang added a commit to leofang/cupy that referenced this pull request Aug 10, 2026
The FlexCI test pipelines (.pfnci/{linux,windows}) now pip-install the
wheel artifact produced by the GHA build stage (added in cupy#10049) instead
of source-building CuPy in every test job.

Wheel fetch (linux/tests/actions/build.sh + windows/test.ps1):

- Derives the artifact suffix ci-guard.sh emits: `pr<N>-<head sha>` for
  /test-driven pull_request builds, `<sha>` post-merge. On Linux,
  PULL_REQUEST is forwarded by run.sh from FlexCI; on Windows,
  IsPullRequestTest + GetPullRequestNumber parse FLEXCI_BRANCH
  (refs/pull/<N>/head).

- Producer pin: the wheel must come from a successful ci.yml run
  (workflow file by path) at the tested SHA under the expected event
  (pull_request for /test, push post-merge), via GET
  /actions/workflows/ci.yml/runs. `path` and `event` are API metadata
  the artifact producer can't forge -- a sibling workflow that shares
  the tested head SHA cannot satisfy this filter.

- Download by immutable artifact ID via GET /actions/artifacts/<id>/zip,
  not by name, so re-run-attempt artifacts can't shadow the intended
  one. Windows routes the binary download through cmd.exe /c to bypass
  PowerShell's UTF-16 stdout redirect (which would corrupt the zip).

- FlexCI's checkout may be the tested commit directly, or a merge of
  that commit into its base -- both are tried (HEAD, then HEAD^2).

Token + tools plumbing:

- linux/run.sh forwards CUPY_CI_GITHUB_TOKEN into the test container.
  Name-only --env + ${VAR+x} set-check keeps the token value out of
  the `set -x` trace.

- generate.py + Linux test Dockerfiles install a pinned `gh` CLI
  (v2.95.0).

CUPY_CI_GITHUB_TOKEN is provisioned FlexCI-side as a fine-grained PAT
(Actions:Read on cupy/cupy). FlexCI targets whose (CUDA, Python) tuple
is outside the wheel-build matrix fail at the fetch step by design,
until the matrix expands.
@leofang

leofang commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

/test skip

@leofang

leofang commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

Let's merge and get it tested in #10058.

@leofang
leofang merged commit 9d5fb2a into cupy:main Aug 10, 2026
65 checks passed
@leofang
leofang deleted the ci/build-stage-shared-infra branch August 10, 2026 08:42
leofang added a commit that referenced this pull request Aug 10, 2026
The hygiene job clears the stale ci:triggered label with
`gh api --method DELETE .../issues/<pr>/labels/ci:triggered` using
GITHUB_TOKEN, but was scoped to `issues: write`. Modifying labels on a pull
request requires `pull-requests: write` (issues:write only authorizes labels
on issues), so the call fails with HTTP 403 "Resource not accessible by
integration" on every pull_request_target (synchronize/reopen) run.

pull_request_target runs the base branch's copy of the workflow, so this
was not exercised until the ci-trigger workflow reached main in #10049.
leofang added a commit that referenced this pull request Aug 11, 2026
CI: build cupy-cuda{12,13}x wheels on GH-hosted runners, gated behind /test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat:test Test code / CI ci:triggered Triggers the CI build pipeline (bounced automatically by /test) prio:high to-be-backported Pull-requests to be backported to stable branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants