[CI] Set up a nightly pipeline to test XGBoost with latest CCCL/RMM#11863
Conversation
|
Great. Both of the failures seen right now can be fixed. The thrust::pair issues indicate a missing header, and you should migrate to cuda::std::pair as well. The RMM issues are from a movement in headers, to Is it possible to run these as optional PR CI jobs too (failure is okay)? |
|
@trivialfis Hi, would you be okay if we added optional CI jobs to pull requests to check for CCCL compatibility? The CI job will be hosted as a separate workflow apart from other jobs. |
Could you please help me catch up to speed? What happened with that conversation and what's changed? Why the change of plan? |
|
Here is the detailed plan for integration testing. I will be on vacation between Dec 13 - Jan 10 and wanted to get a CI running before heading off. If I were to try setting up CI pipelines in CCCL and cuML, I won't be able to get it done in time. |
|
|
||
| set -euo pipefail | ||
|
|
||
| tmpfile="$(mktemp /tmp/abc-script.XXXXXX)" |
There was a problem hiding this comment.
Would like to work on simplifying the CI one day. The amount of bash scripts and their complexity is getting a bit too much now. Also, the failure rate of the CI is relatively high at the moment.
|
Thanks everyone! 🙏 |
4 participants
Example CI run: https://github.com/dmlc/xgboost/actions/runs/20156973678, where the pipeline detects breakage with CCCL 3.2.0 RC1 and RMM 26.02 nightly.
There are two jobs in the pipeline:
test-latest-ccclbuilds XGBoost with the latest version of CCCL (RC included). RMM is turned off for this job. (This is to avoid incompatibilities between latest CCCL and latest RMM.)test-latest-rmmbuilds XGBoost with the latest development version of RMM. The job uses the same CCCL version as rest of Rapids.cc @csadorf @jakirkham @bdice
The pipeline will run on a nightly basis. In addition, it's also possible to manually trigger the pipeline via the workflow dispatch.
@trivialfis Eventually this testing will be moved to CCCL and cuML for the sake of better visibility. For now, we host it in the XGBoost repo. It is set up as a nightly pipeline so that it doesn't block daily development.