@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

I used GitHub Actions to precompile DeepSpeed, but after upgrading to v0.16.4, the precompilation failed. The repository is here: https://github.com/AlongWY/deepspeed_wheels

And the compile result here show that tranformer inference is not support cuda 12.6, which is also shown in #6709.

@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

I used GitHub Actions to precompile DeepSpeed, but after upgrading to v0.16.4, the precompilation failed. The repository is here: https://github.com/AlongWY/deepspeed_wheels

And the compile result here show that tranformer inference is not support cuda 12.6, which is also shown in #6709.

@AlongWY - just so I follow, you're compiling in a GitHub action that has cuda installed but no GPU (so that's why you are hitting the CPU only path here) but with DS_ACCELERATOR=cuda? Just wanted to make sure I understood the repo correctly?

Also the error you shared is only on Windows (that we've observed) but your action appears to be running on Ubuntu, is that right?

Yes, the GitHub Action is compiling in an Ubuntu environment with CUDA installed but without a GPU.
The GitHub Action runs only on Ubuntu, but the error still occurs.

Yes, the GitHub Action is compiling in an Ubuntu environment with CUDA installed but without a GPU. The GitHub Action runs only on Ubuntu, but the error still occurs.

Thanks @AlongWY - can you share the error that you were seeing in this environment? Also if you have triton installed in the environment or not?

Yes, the triton has been installed in the environment.

Error for triton

Error for type

Any update?

Any update?

@AlongWY apologies for the delay. This overall looks good to me. I will appreciate a couple more clarifications.

1. I am curious why accelerator detection logic is incorrectly identifying as cuda env.. Or are you hardcoding DS_ACCELERATOR=cuda` in your code?
2. Are you aligned with the suggested code change?

@sfc-gh-truwase Hi, i have update the code.

For the two questions:

1. I use GitHub Actions to automatically build wheel packages for different environments, so I manually set the environment variable DS_ACCELERATOR=cuda.
2. I have aligned with the suggested code change.