👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

Hi! @mgoin thoughts on this:

Note: I may be misinterpreting the purpose of ENABLE_CUTLASS_MOE_SM100 as it may be being used not literally to mean only SM100. However, if this is the case, this imprecision should be addressed to fix the code and reduce confusion.

link #26571 for visibility

@johnnynunez - looks like this recent dependency issue with apache-tvm-ffi. The main branch build is also failing. From the CI job logs:

No solution found when resolving dependencies:
[2025-10-29T05:55:33Z] 1.626   ╰─▶ Because there is no version of apache-tvm-ffi==0.1.0b15 and
[2025-10-29T05:55:33Z] 1.626       flashinfer-python==0.4.1 depends on apache-tvm-ffi==0.1.0b15, we can
[2025-10-29T05:55:33Z] 1.626       conclude that flashinfer-python==0.4.1 cannot be used.
[2025-10-29T05:55:33Z] 1.626       And because you require flashinfer-python==0.4.1, we can conclude that
[2025-10-29T05:55:33Z] 1.626       your requirements are unsatisfiable.
[2025-10-29T05:55:33Z] 1.626
[2025-10-29T05:55:33Z] 1.626       hint: `apache-tvm-ffi` was requested with a pre-release marker (e.g.,
[2025-10-29T05:55:33Z] 1.626       apache-tvm-ffi==0.1.0b15), but pre-releases weren't enabled (try:
[2025-10-29T05:55:33Z] 1.626       `--prerelease=allow`)

mmm maybe he has to merge main to his PR due that apache tvm ffi now has stable release

mmm maybe he has to merge main to his PR due that apache tvm ffi now has stable release

This also happens in main, because of dependency on flashinfer==0.4.1 that triggers this apache dependency. There is a new version of flashinfer==0.5.0, I wonder if they should use that instead (assuming no breaking changes).

mmm maybe he has to merge main to his PR due that apache tvm ffi now has stable release

This also happens in main, because of dependency on flashinfer==0.4.1 that triggers this apache dependency. There is a new version of flashinfer==0.5.0, I wonder if they should use that instead (assuming no breaking changes).

there is an open PR. Be patient

On Spark, you need two things:

1. Apply the changes from this PR.
2. Set environment variables before building VLLM (this is also essential, otherwise cmake will not pick up correct architecture):

export TORCH_CUDA_ARCH_LIST=12.0f export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas

I tried to export this two env then compile

but still encounter error.

NotImplementedError: No compiled get_cutlass_moe_mm_data: no cutlass_scaled_mm kernel for CUDA device capability: 121. Required capability: 90 or 100

Is this means we cannot run moe model on spark now? @eugr

I tried to export this two env then compile

Have you applied the changes from this PR to CMakeLists.txt?

I tried to export this two env then compile

Have you applied the changes from this PR to CMakeLists.txt?

sure. And I successfully run qwen3-32b. But moe models are keep failing

I tried to export this two env then compile

Have you applied the changes from this PR to CMakeLists.txt?

sure. And I successfully run qwen3-30b-a3b. But moe models are keep failing

di you tried new pytorch 2.9.1 and triton 3.5.1? That fix some issues for spark related to triton kernels

di you tried new pytorch 2.9.1 and triton 3.5.1? That fix some issues for spark related to triton kernels

I used pytorch 2.9.0. But I don't think thats the case cause the cutlass_scaled_fp4_mm is a cuda kernel? And maybe it seems we don't get some thing like "cutlass_scaled_fp4_mm_sm120f" or "cutlass_scaled_fp4_mm_sm121" in vllm, which related to spark's arch.

di you tried new pytorch 2.9.1 and triton 3.5.1? That fix some issues for spark related to triton kernels

I used pytorch 2.9.0. But I don't think thats the case cause the cutlass_scaled_fp4_mm is a cuda kernel? And maybe it seems we don't get some thing like "cutlass_scaled_fp4_mm_sm120f" or "cutlass_scaled_fp4_mm_sm121" in vllm, which related to spark's arch.

yes, there is a lack of support for spark right now, we are working on it

di you tried new pytorch 2.9.1 and triton 3.5.1? That fix some issues for spark related to triton kernels

I used pytorch 2.9.0. But I don't think thats the case cause the cutlass_scaled_fp4_mm is a cuda kernel? And maybe it seems we don't get some thing like "cutlass_scaled_fp4_mm_sm120f" or "cutlass_scaled_fp4_mm_sm121" in vllm, which related to spark's arch.

yes, there is a lack of support for spark right now, we are working on it

@johnnynunez im interested in following developments with spark support, where can I do that?

Since there have been some Spark related questions here, I'm wondering if I can please add one:

It seems that doing python -m build --wheel fails on Spark because when it creates the isolated environment for building (based on the toml file) it pulls in the CPU-only version of torch, not the CUDA version, even if you set VLLM_TARGET_DEVICE=cuda. The only way I could get it to successfully build was doing python -m build --wheel --no-isolation because my local install of torch is the CUDA enabled one.

Does anyone here know how I can fix this, such that the isolated build environment pulls in the correct CUDA-enabled torch instead of the CPU-only variant?

it pulls in the CPU-only version of torch

Run python use_existing_torch.py before building.

it pulls in the CPU-only version of torch

Run python use_existing_torch.py before building.

That doesn't work though, because all python use_existing_torch.py does is just erase torch from pyproject.toml. Then when you run python -m build --wheel the isolated environment fails immediately because it doesn't download any torch at all, neither the cpu or cuda one, so you get an ImportError "no module torch".

python -m build -wheel creates an isolated environment for the build, completely separate from the local environment, and it pulls everything in from the toml, so basically just like how uv run works. So if you remove torch from the toml, the isolated build will fail as there's no torch at all anymore.

Any reason you can't use --no-build-isolation?

Any reason you can't use --no-build-isolation?

I can, in which case it works just fine, however, that is only because my local Python environment already mirrors the contents of the toml file. The point of an isolated build is that it is essentially running uv run python -m build --wheel under the bonnet, and this is super useful because it means I don't have to worry if my local python environment matches what's necessary to build. I could have a different local cuda or torch version, whatever. But I can still build vllm any time by creating an isolated environment on the fly that always matches the toml. It's super useful and is the correct way to build complex packages as there is guaranteed no version mismatches.

Only on the DGX Spark it's not working correctly because the toml is pointing to the CPU-only install for torch, and I'm wondering how we change that to be the cuda version instead. As it stands currently, I believe it's a legitimate bug, and should affect any unix box, not just the Spark, but I don't have another unix box with a GPU to know for certain.

I could have a different local cuda or torch version, whatever.

Won't venv give you that? venv works well with uv, just use uv venv to create it and then activate as usual.

I could have a different local cuda or torch version, whatever.

Won't venv give you that? venv works well with uv, just use uv venv to create it and then activate as usual.

venv is what uv uses under the bonnet, same with python build. Unfortunately, the toml file in vllm isn't telling the venv system to use the cuda variant of torch. You need to explicitly tell it to do that, like how Nemo-RL does it: https://github.com/NVIDIA-NeMo/RL/blob/2951ce37ae1e568ab5ff063997da478a85551dfa/pyproject.toml#L185

@Jonahcb - looks like you need to merge main into your PR for checks to pass...

cc @mgoin

Can I ask why sm110 is being excluded? It has all of the scaled MM ops of sm100.

@Jonahcb could you resolve the comments? Thanks

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Jonahcb.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork