Skip to content

ci: disable sccache preprocessor cache mode - #141

Merged
atmnp merged 3 commits into
NVIDIA:mainfrom
leofang:leofang/disable-sccache-preprocessor-mode
Jun 24, 2026
Merged

atmnp merged 3 commits into
NVIDIA:mainfrom
leofang:leofang/disable-sccache-preprocessor-mode

Conversation

@leofang

@leofang leofang commented Jun 16, 2026

Copy link
Copy Markdown
Member

Mirrors NVIDIA/cuda-python#2221 — see NVIDIA/cuda-python#2220 for the full investigation.

Same root cause applies here: pip's per-run /tmp/pip-build-env-<RANDOM>/... overlay path leaks into -I flags during the cibuildwheel build, so sccache's preprocessor cache key is randomized every run. In cuSIMT the symptom is milder than in cuda-python (CMake doesn't propagate every overlay path through to gcc — last main run had 9 hits / 25 misses) but it's still net overhead.

LLVM 7 / modern LLVM builds (.github/workflows/build-llvm.yml + ci/setup-sccache.sh) are unaffected: they use the S3 backend with [cache.s3.preprocessor_cache_mode] and don't go through cibuildwheel.

The preprocessor cache mode produces minimal/zero hits across CI runs
because pip's per-run build-isolation overlay path leaks into compiler
flags. See NVIDIA/cuda-python#2220 for the full investigation.

LLVM 7 / ToT builds are unaffected — they use the S3 backend configured
via ci/setup-sccache.sh and don't go through cibuildwheel.
@leofang leofang self-assigned this Jun 16, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@leofang

leofang commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

/ok to test 1708603

@github-actions

This comment has been minimized.

@leofang
leofang marked this pull request as ready for review June 16, 2026 05:08
@atmnp

atmnp commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

/ok to test e7372c8

@atmnp
atmnp enabled auto-merge (squash) June 23, 2026 23:47
@atmnp
atmnp merged commit 1808006 into NVIDIA:main Jun 24, 2026
126 checks passed
@github-actions

Copy link
Copy Markdown
Doc Preview CI
Preview removed because the pull request was closed or merged.

@leofang
leofang deleted the leofang/disable-sccache-preprocessor-mode branch June 24, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants