Skip to content

Fix sccache summary to count CUDA sub-tool cache hits/misses - #8858

Merged
leofang merged 2 commits into
NVIDIA:mainfrom
leofang:fix/sccache-summary-cuda-subcounts
May 7, 2026
Merged

leofang merged 2 commits into
NVIDIA:mainfrom
leofang:fix/sccache-summary-cuda-subcounts

Conversation

@leofang

@leofang leofang commented May 7, 2026

Copy link
Copy Markdown
Member

Summary

The sccache execution summary inflates the reported cache hit rate because CUDA sub-tool compilations are invisible to the metric.

The rapidsai/sccache fork tracks CUDA compilation sub-phases under separate language keys in its JSON stats:

  • "CUDA" — nvcc driver pass
  • "CUDA (Device code)" — cudafe++ pass
  • "PTX" — cicc pass
  • "CUBIN" — ptxas pass

The summary script only counted hits from {"C/C++", "CUDA"} and used compile_requests as the denominator. Since compile_requests only counts top-level nvcc invocations, sub-tool cache misses were invisible to both numerator and denominator — inflating the reported rate.

Concrete example from run 25468002551, job CY (cudax nvcc GCC / [CTK13.2 GCC15 C++20] Build):

Before (bug) After (fix)
Hits 707 707
Denominator 711 (compile_requests) 763 (hits + misses)
Hidden misses 56 (CUBIN=24, PTX=24, CUDA Device code=4, CUDA=4) 0
Reported rate 99% 92%

This PR:

  • Adds "CUDA (Device code)", "PTX", and "CUBIN" to the language filter
  • Switches the denominator from compile_requests to hits + misses (matching sccache's own "Cache hits rate" calculation)

Related: NVIDIA/numba-cuda#878 (same fix for numba-cuda's sccache summary action)

Test plan

  • Reproduced locally with mock JSON matching the real CI run data — old logic gives 99%, new logic gives 92%
  • The fix will be validated on the next CI run that produces sccache stats

The rapidsai/sccache fork tracks CUDA compilation sub-phases under
separate language keys in the JSON stats: "CUDA" (nvcc driver),
"CUDA (Device code)" (cudafe++), "PTX" (cicc), and "CUBIN" (ptxas).

The summary script only counted hits from "C/C++" and "CUDA", and
used compile_requests as the denominator.  Since compile_requests
only counts top-level nvcc invocations, sub-tool cache misses were
invisible to both numerator and denominator, inflating the reported
hit rate (e.g. 99% when the true rate was ~93%).

Fix both issues:
- Include all CUDA-related language keys in the filter
- Use hits + misses as the denominator (matching sccache's own
  "Cache hits rate" calculation)
@copy-pr-bot

copy-pr-bot Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-project-automation github-project-automation Bot moved this to Todo in CCCL May 7, 2026
@cccl-authenticator-app cccl-authenticator-app Bot moved this from Todo to In Progress in CCCL May 7, 2026
@leofang
leofang marked this pull request as ready for review May 7, 2026 03:45
@leofang
leofang requested a review from a team as a code owner May 7, 2026 03:45
@leofang
leofang requested a review from jrhemstad May 7, 2026 03:45
@leofang

leofang commented May 7, 2026

Copy link
Copy Markdown
Member Author

/ok to test 0420eda

@cccl-authenticator-app cccl-authenticator-app Bot moved this from In Progress to In Review in CCCL May 7, 2026
@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

🥳 CI Workflow Results

🟩 Finished in 3h 28m: Pass: 100%/500 | Total: 5d 23h | Max: 3h 27m | Hits: 65%/1125770

See results here.

@leofang
leofang merged commit 7560382 into NVIDIA:main May 7, 2026
522 checks passed
@leofang
leofang deleted the fix/sccache-summary-cuda-subcounts branch May 7, 2026 15:15
@leofang

leofang commented May 7, 2026

Copy link
Copy Markdown
Member Author

Thanks, Allison!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants