[None][perf] Adjust select_alltoall_method_type. #8950

bobboli · 2025-11-05T16:44:44Z

Adjust select_alltoall_method_type for better OOTB perf:

Use MNNVL AlltoAll if possible, without considering ep_size and top_k.
Use MnnvlThroughput instead of MnnvlLatency by default.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

Summary by CodeRabbit

Refactor
- Updated AlltoAll distributed execution logic with refined enabling conditions based on parallelism configuration.
- Changed default MoE backend preference from latency-oriented to throughput-oriented for improved overall performance.
- Simplified EP size dependency checks for AlltoAll optimization enablement.

bobboli · 2025-11-05T16:44:56Z

/bot run

tensorrt-cicd · 2025-11-05T16:50:41Z

PR_Github #23662 [ run ] triggered by Bot. Commit: 70dd84d

coderabbitai · 2025-11-05T16:51:39Z

📝 Walkthrough

Walkthrough

Both fused MoE modules update AlltoAll gating logic for method selection, replacing attention data parallelism checks with direct data parallelism size conditions and simplifying EP size validation. Default backend selection changes from "mnnvllatency" to "mnnvlthroughput" in both files.

Changes

Cohort / File(s)	Change Summary
AlltoAll gating and backend defaults `tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py`, `tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py`	Updated AlltoAll enabling logic: replaced `enable_attention_dp` check with `dp_size == 1` condition to disable when data parallelism absent; changed TP size restriction from `tp_size == 1` to `moe_tp_size != 1`; removed `moe_ep_size > routing_method.experts_per_token` conditional gating, now AlltoAll depends only on MnnvlMemory support; changed default `moe_alltoall_backend` from "mnnvllatency" to "mnnvlthroughput"; added documentation clarifying MNNVL preference independent of EP size relationship.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Review consistency of gating logic changes across both files (DP/TP size condition inversions differ in exact comparison operators; verify correctness)
Validate removal of EP size check does not unintentionally broaden AlltoAll eligibility in unexpected scenarios
Confirm default backend change rationale and impact on throughput vs. latency trade-offs

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The PR description is vague and incomplete; it lacks detailed explanations of the changes and test coverage information is entirely missing.	Fill in the Description section with detailed explanations of why changes were made, complete the Test Coverage section with specific tests, and check the PR Checklist items.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly describes the main change: adjusting the select_alltoall_method_type function with a focus on performance improvements.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e57d83c and 70dd84d.

📒 Files selected for processing (2)

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py (2 hunks)
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Use only spaces, no tabs; indent with 4 spaces.

Files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

**/*.py

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.py: Python code must target Python 3.8+.
Indent Python code with 4 spaces; do not use tabs.
Maintain module namespace when importing; prefer 'from package.subpackage import foo' then 'foo.SomeClass()' instead of importing the class directly.
Python filenames should be snake_case (e.g., some_file.py).
Python classes use PascalCase names.
Functions and methods use snake_case names.
Local variables use snake_case; prefix 'k' for variables that start with a number (e.g., k_99th_percentile).
Global variables use upper SNAKE_CASE prefixed with 'G' (e.g., G_MY_GLOBAL).
Constants use upper SNAKE_CASE (e.g., MY_CONSTANT).
Avoid shadowing variables from an outer scope.
Initialize all externally visible members of a class in the constructor.
Prefer docstrings for interfaces that may be used outside a file; comments for in-function or file-local interfaces.
Use Google-style docstrings for classes and functions (Sphinx-parsable).
Document attributes and variables inline so they render under the class/function docstring.
Avoid reflection when a simpler, explicit approach suffices (e.g., avoid dict(**locals()) patterns).
In try/except, catch the most specific exceptions possible.
For duck-typing try/except, keep the try body minimal and use else for the main logic.

Files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

**/*.{cpp,cxx,cc,h,hpp,hh,hxx,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Prepend the NVIDIA Apache-2.0 copyright header with current year to the top of all source files (e.g., .cpp, .h, .cu, .py).

Files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

🧠 Learnings (13)

📓 Common learnings

Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu:118-127
Timestamp: 2025-08-09T20:57:04.084Z
Learning: In the CUTLASS MoE finalize fusion implementation (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu), when setting `fused_finalize_epilogue.stride_final_output` with shape `(hidden_size, num_output_tokens, 1)`, the `num_rows_in_final_output` should be set to `num_output_tokens` (not `hidden_size`) because of a swap+transpose operation that maps rows of the output tensor to `hidden_size` and columns to `num_output_tokens`.

Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 7104
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1475-1480
Timestamp: 2025-08-21T02:39:12.009Z
Learning: The min latency mode functionality in TensorRT-LLM MOE kernels (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu) is deprecated and no longer being maintained/updated, as confirmed by djns99. Bug reports and optimization suggestions for the computeStridesTmaWarpSpecializedLowLatencyKernel and related min latency code paths should be deprioritized.

Learnt from: timlee0212
Repo: NVIDIA/TensorRT-LLM PR: 6886
File: tensorrt_llm/_torch/models/modeling_deepseekv3.py:0-0
Timestamp: 2025-08-14T06:36:40.701Z
Learning: In DeepSeek V3 model (tensorrt_llm/_torch/models/modeling_deepseekv3.py), the disagreement between AllReduce.__init__ guard and _compute_mlp_tp_size logic for MNNVL usage is expected by design. The AllReduce component and MLP TP-size computation intentionally use different criteria for MNNVL availability decisions.

Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/thop/allreduceOp.cpp:352-446
Timestamp: 2025-09-23T15:12:38.312Z
Learning: In TensorRT-LLM NCCL device allreduce implementation (cpp/tensorrt_llm/thop/allreduceOp.cpp), the goto pattern in runNCCLAllReduceDeviceFusion is intentionally used for future extensibility, allowing multiple switch cases to fallback to the default handler. While not aesthetically ideal, this pattern supports adding more fusion cases later that can reuse the same fallback logic.

Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1198-1209
Timestamp: 2025-08-08T22:03:40.707Z
Learning: In the CUTLASS MoE kernels (cpp/tensorrt_llm/cutlass_extensions), when `layout_info.fusion` is set to `TmaWarpSpecializedGroupedGemmInput::EpilogueFusion::FINALIZE`, the `router_scales` parameter must be non-null by design. The fused finalize kernel epilogue does not perform nullptr checks and requires valid router scales to function correctly. This is an implicit contract that callers must satisfy when enabling the FINALIZE fusion mode.

📚 Learning: 2025-08-09T20:57:04.084Z

Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu:118-127
Timestamp: 2025-08-09T20:57:04.084Z
Learning: In the CUTLASS MoE finalize fusion implementation (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_tma_warp_specialized_input.cu), when setting `fused_finalize_epilogue.stride_final_output` with shape `(hidden_size, num_output_tokens, 1)`, the `num_rows_in_final_output` should be set to `num_output_tokens` (not `hidden_size`) because of a swap+transpose operation that maps rows of the output tensor to `hidden_size` and columns to `num_output_tokens`.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-21T02:39:12.009Z

Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 7104
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1475-1480
Timestamp: 2025-08-21T02:39:12.009Z
Learning: The min latency mode functionality in TensorRT-LLM MOE kernels (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu) is deprecated and no longer being maintained/updated, as confirmed by djns99. Bug reports and optimization suggestions for the computeStridesTmaWarpSpecializedLowLatencyKernel and related min latency code paths should be deprioritized.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-08T22:03:40.707Z

Learnt from: sklevtsov-nvidia
Repo: NVIDIA/TensorRT-LLM PR: 3294
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1198-1209
Timestamp: 2025-08-08T22:03:40.707Z
Learning: In the CUTLASS MoE kernels (cpp/tensorrt_llm/cutlass_extensions), when `layout_info.fusion` is set to `TmaWarpSpecializedGroupedGemmInput::EpilogueFusion::FINALIZE`, the `router_scales` parameter must be non-null by design. The fused finalize kernel epilogue does not perform nullptr checks and requires valid router scales to function correctly. This is an implicit contract that callers must satisfy when enabling the FINALIZE fusion mode.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-09-23T15:12:38.312Z

Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/thop/allreduceOp.cpp:352-446
Timestamp: 2025-09-23T15:12:38.312Z
Learning: In TensorRT-LLM NCCL device allreduce implementation (cpp/tensorrt_llm/thop/allreduceOp.cpp), the goto pattern in runNCCLAllReduceDeviceFusion is intentionally used for future extensibility, allowing multiple switch cases to fallback to the default handler. While not aesthetically ideal, this pattern supports adding more fusion cases later that can reuse the same fallback logic.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-14T23:23:27.449Z

Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4010-4012
Timestamp: 2025-08-14T23:23:27.449Z
Learning: For MOE (Mixture of Experts) code reviews in TensorRT-LLM, avoid repeatedly suggesting finalize fusion validation checks and safety assertions. The user djns99 has indicated these suggestions are repetitive and unwanted across multiple MOE-related changes.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-14T06:36:40.701Z

Learnt from: timlee0212
Repo: NVIDIA/TensorRT-LLM PR: 6886
File: tensorrt_llm/_torch/models/modeling_deepseekv3.py:0-0
Timestamp: 2025-08-14T06:36:40.701Z
Learning: In DeepSeek V3 model (tensorrt_llm/_torch/models/modeling_deepseekv3.py), the disagreement between AllReduce.__init__ guard and _compute_mlp_tp_size logic for MNNVL usage is expected by design. The AllReduce component and MLP TP-size computation intentionally use different criteria for MNNVL availability decisions.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-19T12:45:11.997Z

Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 7033
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:0-0
Timestamp: 2025-08-19T12:45:11.997Z
Learning: In tensorrt_llm/_torch/pyexecutor/model_engine.py, DoRA (Delta Orthogonal Rank Adaptation) functionality was removed from the PyTorch flow to eliminate issues with inverted DoRA detection logic. The original is_dora condition was checking if scaling_vec_pointer == 0, which was potentially incorrect.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-08-19T03:35:20.866Z

Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4616-4626
Timestamp: 2025-08-19T03:35:20.866Z
Learning: In the MOE profiler TMA workspace preparation (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu), the overlapping of TMA WS regions for NONE and FINALIZE variants is deliberate design to save memory space, as confirmed by djns99. The comment "reuse the same pointers to save space" reflects this intentional behavior.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-09-19T21:28:13.751Z

Learnt from: jhaotingc
Repo: NVIDIA/TensorRT-LLM PR: 7856
File: cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp:159-166
Timestamp: 2025-09-19T21:28:13.751Z
Learning: In TensorRT-LLM blockScaleMoe routing (cpp/tensorrt_llm/kernels/trtllmGenKernels/blockScaleMoe/runner.cu), the DeepSeek routing method performs reinterpret_cast<float*>(routingLogits) at line 89, which could cause issues if routing_logits are BF16. However, Qwen3-FP8 models use RenormalizeNaive routing method and are not affected by this dtype casting issue.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py

📚 Learning: 2025-08-26T09:37:10.463Z

Learnt from: jiaganc
Repo: NVIDIA/TensorRT-LLM PR: 7031
File: tensorrt_llm/bench/dataclasses/configuration.py:90-104
Timestamp: 2025-08-26T09:37:10.463Z
Learning: In TensorRT-LLM's bench configuration, the `get_pytorch_perf_config()` method returns `self.pytorch_config` which is a Dict[str, Any] that can contain default values including `cuda_graph_config`, making the fallback `llm_args["cuda_graph_config"]` safe to use.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-09-29T15:14:28.503Z

Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 8063
File: tensorrt_llm/lora_manager.py:1080-1112
Timestamp: 2025-09-29T15:14:28.503Z
Learning: In tensorrt_llm/lora_manager.py, when calculating part_sizes for attn_qkv fused LoRA modules, the sizes are correctly multiplied by tp_size because model_config.num_heads and model_config.num_kv_heads are already divided by tp_size (per-TP-rank values), so multiplication is needed to get the original full concatenated dimension size. The interleave_fused_lora_weights_for_tp function provides proper validation with asserts for total size and TP divisibility.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

📚 Learning: 2025-09-29T15:14:28.503Z

Learnt from: amitz-nv
Repo: NVIDIA/TensorRT-LLM PR: 8063
File: tensorrt_llm/lora_manager.py:1080-1112
Timestamp: 2025-09-29T15:14:28.503Z
Learning: In tensorrt_llm/lora_manager.py, when calculating part_sizes for attn_qkv fused LoRA modules, the sizes are correctly multiplied by tp_size because model_config.num_heads and model_config.num_kv_heads are already divided by tp_size (per-TP-rank values), so multiplication is needed to get the original full concatenated dimension size. The interleave_fused_lora_weights_for_tp function provides proper validation.

Applied to files:

tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

🧬 Code graph analysis (2)

tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py (3)

tensorrt_llm/mapping.py (1)

dp_size (222-223)

tensorrt_llm/_torch/modules/fused_moe/interface.py (1)

AlltoallMethodType (26-34)

tensorrt_llm/_mnnvl_utils.py (2)

MnnvlMemory (53-338)

supports_mnnvl (332-338)

tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py (3)

tensorrt_llm/mapping.py (1)

dp_size (222-223)

tensorrt_llm/_torch/modules/fused_moe/interface.py (1)

AlltoallMethodType (26-34)

tensorrt_llm/_mnnvl_utils.py (2)

MnnvlMemory (53-338)

supports_mnnvl (332-338)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: Pre-commit Check

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

tensorrt-cicd · 2025-11-05T20:15:40Z

PR_Github #23662 [ run ] completed with state SUCCESS. Commit: 70dd84d
/LLM/main/L0_MergeRequest_PR pipeline #17803 completed with status: 'FAILURE'

bobboli · 2025-11-06T16:28:52Z

/bot run

tensorrt-cicd · 2025-11-06T16:35:06Z

PR_Github #23755 [ run ] triggered by Bot. Commit: 9600dc1

tensorrt-cicd · 2025-11-06T20:37:50Z

PR_Github #23755 [ run ] completed with state SUCCESS. Commit: 9600dc1
/LLM/main/L0_MergeRequest_PR pipeline #17880 completed with status: 'FAILURE'

tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py

bobboli · 2025-11-07T17:44:10Z

/bot run

tensorrt-cicd · 2025-11-07T17:50:06Z

PR_Github #23866 [ run ] triggered by Bot. Commit: 0cf9883

tensorrt-cicd · 2025-11-07T19:53:12Z

PR_Github #23866 [ run ] completed with state SUCCESS. Commit: 0cf9883
/LLM/main/L0_MergeRequest_PR pipeline #17967 completed with status: 'FAILURE'

bobboli · 2025-11-10T02:42:31Z

/bot run

tensorrt-cicd · 2025-11-10T02:48:08Z

PR_Github #23941 [ run ] triggered by Bot. Commit: fd09ced

tensorrt-cicd · 2025-11-10T04:01:15Z

PR_Github #23941 [ run ] completed with state SUCCESS. Commit: fd09ced
/LLM/main/L0_MergeRequest_PR pipeline #18027 completed with status: 'FAILURE'

bobboli · 2025-11-10T05:08:20Z

/bot run

tensorrt-cicd · 2025-11-10T05:13:59Z

PR_Github #23957 [ run ] triggered by Bot. Commit: 687468c

tensorrt-cicd · 2025-11-10T10:03:34Z

PR_Github #23957 [ run ] completed with state SUCCESS. Commit: 687468c
/LLM/main/L0_MergeRequest_PR pipeline #18040 completed with status: 'FAILURE'

bobboli · 2025-11-10T10:15:41Z

/bot run --reuse-test

tensorrt-cicd · 2025-11-18T08:55:51Z

PR_Github #24884 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-18T08:55:54Z

PR_Github #24856 [ run ] completed with state ABORTED. Commit: 9e76e0d
LLM/main/L0_MergeRequest_PR #18763 (Blue Ocean) completed with status: ABORTED

bobboli · 2025-11-18T09:13:43Z

/bot run

tensorrt-cicd · 2025-11-18T09:19:48Z

PR_Github #24890 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-18T09:19:50Z

PR_Github #24884 [ run ] completed with state ABORTED. Commit: 70599b4
LLM/main/L0_MergeRequest_PR #18787 (Blue Ocean) completed with status: ABORTED

tensorrt-cicd · 2025-11-18T12:48:28Z

PR_Github #24890 [ run ] completed with state SUCCESS. Commit: 70599b4
/LLM/main/L0_MergeRequest_PR pipeline #18793 completed with status: 'FAILURE'

bobboli · 2025-11-18T13:07:28Z

/bot run --reuse-test

tensorrt-cicd · 2025-11-18T13:12:58Z

PR_Github #24910 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-19T00:18:06Z

PR_Github #24910 [ run ] completed with state SUCCESS. Commit: 70599b4
/LLM/main/L0_MergeRequest_PR pipeline #18811 completed with status: 'FAILURE'

bobboli · 2025-11-19T03:25:11Z

/bot run

tensorrt-cicd · 2025-11-19T03:31:23Z

PR_Github #24979 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-19T05:56:15Z

PR_Github #24979 [ run ] completed with state FAILURE. Commit: 70599b4
/LLM/main/L0_MergeRequest_PR pipeline #18869 completed with status: 'FAILURE'

bobboli · 2025-11-19T06:46:24Z

/bot run --reuse-test

tensorrt-cicd · 2025-11-19T06:52:09Z

PR_Github #24996 [ run ] triggered by Bot. Commit: 70599b4

bobboli · 2025-11-19T08:28:22Z

/bot kill

bobboli · 2025-11-19T08:28:31Z

/bot run --reuse-test

tensorrt-cicd · 2025-11-19T08:34:45Z

PR_Github #25022 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-19T08:35:15Z

PR_Github #25025 [ kill ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-19T08:35:22Z

PR_Github #25022 [ run ] completed with state ABORTED. Commit: 70599b4

tensorrt-cicd · 2025-11-19T08:35:52Z

PR_Github #25025 [ kill ] completed with state SUCCESS. Commit: 70599b4
Successfully killed previous jobs for commit 70599b4

bobboli · 2025-11-19T08:41:00Z

/bot run --reuse-test

tensorrt-cicd · 2025-11-19T08:46:51Z

PR_Github #25027 [ run ] triggered by Bot. Commit: 70599b4

tensorrt-cicd · 2025-11-19T15:43:52Z

PR_Github #25027 [ run ] completed with state SUCCESS. Commit: 70599b4
/LLM/main/L0_MergeRequest_PR pipeline #18909 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

Signed-off-by: Bo Li <[email protected]> Signed-off-by: lkomali <[email protected]>

bobboli marked this pull request as ready for review November 5, 2025 16:44

bobboli requested a review from a team as a code owner November 5, 2025 16:44

bobboli requested a review from QiJune November 5, 2025 16:44

bobboli requested review from WeiHaocheng, dongxuy04 and xxi-nv November 6, 2025 03:16

xxi-nv approved these changes Nov 6, 2025

View reviewed changes

kaiyux reviewed Nov 7, 2025

View reviewed changes

tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py Outdated Show resolved Hide resolved

bobboli force-pushed the select_alltoall branch from 9abaacb to 087eaaf Compare November 7, 2025 06:54

bobboli enabled auto-merge (squash) November 7, 2025 17:41

bobboli mentioned this pull request Nov 8, 2025

[None][feat] Enable EPLB for trtllm-gen and cutlass backend #8886

Merged

1 task

bobboli force-pushed the select_alltoall branch from 0cf9883 to ef81e60 Compare November 10, 2025 02:32

bobboli force-pushed the select_alltoall branch from fd09ced to 687468c Compare November 10, 2025 05:08

bobboli requested a review from lucaslie November 18, 2025 08:51

lucaslie requested review from nzmora-nvidia and suyoggupta November 18, 2025 17:27

lucaslie approved these changes Nov 19, 2025

View reviewed changes

bobboli merged commit d8b0589 into NVIDIA:main Nov 19, 2025
4 of 5 checks passed

bobboli mentioned this pull request Nov 19, 2025

[None][fix] use_workspace_output only supports w4a8_mxfp4_mxfp8 (gpt-oss) for now. #9259

Closed

1 task

lkomali pushed a commit to lkomali/TensorRT-LLM that referenced this pull request Nov 19, 2025

[None][perf] Adjust select_alltoall_method_type. (NVIDIA#8950)

d77a3ef

Signed-off-by: Bo Li <[email protected]> Signed-off-by: lkomali <[email protected]>

[None][perf] Adjust select_alltoall_method_type. #8950

[None][perf] Adjust select_alltoall_method_type. #8950

Uh oh!

Conversation

bobboli commented Nov 5, 2025 • edited by longlee0622 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

kill

skip

reuse-pipeline

Summary by CodeRabbit

Uh oh!

bobboli commented Nov 5, 2025

Uh oh!

tensorrt-cicd commented Nov 5, 2025

Uh oh!

coderabbitai bot commented Nov 5, 2025

Walkthrough

Changes

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

tensorrt-cicd commented Nov 5, 2025

Uh oh!

bobboli commented Nov 6, 2025

Uh oh!

tensorrt-cicd commented Nov 6, 2025

Uh oh!

tensorrt-cicd commented Nov 6, 2025

Uh oh!

Uh oh!

bobboli commented Nov 7, 2025

Uh oh!

tensorrt-cicd commented Nov 7, 2025

Uh oh!

tensorrt-cicd commented Nov 7, 2025

Uh oh!

bobboli commented Nov 10, 2025

Uh oh!

tensorrt-cicd commented Nov 10, 2025

Uh oh!

tensorrt-cicd commented Nov 10, 2025

Uh oh!

bobboli commented Nov 10, 2025

Uh oh!

tensorrt-cicd commented Nov 10, 2025

Uh oh!

tensorrt-cicd commented Nov 10, 2025

Uh oh!

bobboli commented Nov 10, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

bobboli commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

bobboli commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 18, 2025

Uh oh!

tensorrt-cicd commented Nov 19, 2025

Uh oh!

bobboli commented Nov 19, 2025

Uh oh!

tensorrt-cicd commented Nov 19, 2025

Uh oh!

tensorrt-cicd commented Nov 19, 2025

Uh oh!

bobboli commented Nov 19, 2025

Uh oh!

bobboli commented Nov 5, 2025 •

edited by longlee0622

Loading