Skip to content

Conversation

@achartier
Copy link
Collaborator

@achartier achartier commented Oct 13, 2025

Summary by CodeRabbit

  • New Features

    • Added support for SM100 GPU architecture with optimized FP8 rowwise GEMM operations.
  • Tests

    • Enabled FP8 rowwise linear tests to run on Blackwell-based GPUs.

Description

Add FP8 rowwise GEMMs for B200

Test Coverage

test_fp8_rowwise_linear

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025

📝 Walkthrough

Walkthrough

This PR adds SM100 architecture support to the FP8 rowwise GEMM kernels. Changes include enabling SM100 compilation in CMake, introducing SM100-specific kernel templates and dispatch routing logic, implementing the SM100 device GEMM structure, and removing a Blackwell skip decorator from tests.

Changes

Cohort / File(s) Change Summary
Build Configuration
cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt
Updated fb_gemm_src CUDA architecture list from 89 90 120f to 89 90 100f 120f, enabling SM100 kernel instantiation.
Test Updates
tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py
Removed @skip_blackwell decorator and associated import; test remains parameterized with @skip_pre_hopper.
FP8 GEMM Dispatch Layer
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
Added SM100-specific dispatch paths: prepareGemmArgsSm100, genericFp8RowwiseGemmKernelLauncherSm100, dispatchGemmConfigSm100, and dispatchGemmToCutlassSm100 templates; extended dispatchToArch and getConfigs to route SM100 traffic with SM100-specific tile and cluster configurations.
SM100 Kernel Template
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm100.h
New header defining DeviceGemmFp8RowwiseSm100 template structure with complete FP8 (e4m3) rowwise GEMM configuration for SM100, including explicit A/B/C/Output matrix types, rowwise epilogue fusion chain (XScale, WScale, Bias, Accum, Compute stages), collective mainloop, and SM100-only runtime guard.

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant DispatchRouter as dispatchToArch
    participant SM100Path as SM100 Path
    participant SM90Path as SM89/90/120f Path
    
    Caller->>DispatchRouter: Check mSm value
    alt mSm == 100
        DispatchRouter->>SM100Path: Call dispatchGemmToCutlassSm100
        SM100Path->>SM100Path: Select SM100 tile shapes<br/>and cluster config
        SM100Path->>SM100Path: Call dispatchGemmConfigSm100
        SM100Path->>SM100Path: Launch DeviceGemmFp8RowwiseSm100<br/>kernel with SM100 guard
        SM100Path-->>Caller: Return
    else Other SM versions
        DispatchRouter->>SM90Path: Route to existing<br/>SM89/90/120f logic
        SM90Path-->>Caller: Return
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The PR description is largely incomplete relative to the provided template requirements. The "Description" section contains only a single line restating the title ("Add FP8 rowwise GEMMs for B200") without explaining the issue, motivation, or solution. The "Test Coverage" section merely lists "test_fp8_rowwise_linear" without describing what the test validates or how it covers the new code paths. While the PR checklist is present and one item is marked as complete, the critical explanatory sections that should provide context and justification for the changes are minimal and inadequate. The remainder of the description consists primarily of boilerplate GitHub Bot Help documentation. The author should expand the "Description" section to explain the motivation for adding B200/SM100 support, what FP8 rowwise GEMMs provide, and how the implementation achieves this. The "Test Coverage" section should detail which test cases validate the new SM100 code paths and what aspects of the implementation they verify. These additions will provide reviewers with sufficient context to understand the purpose and scope of the changes.
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Title Check ✅ Passed The pull request title "[None][feat] Add FP8 rowwise GEMMs for B200" directly and clearly describes the main objective of the changeset. The changes consistently add SM100/B200-specific implementations for FP8 rowwise GEMM operations across multiple files: CMakeLists.txt adds SM100 CUDA architecture support, the template header files introduce SM100-specific dispatch paths and kernels, and the test decorator is updated to enable the test on relevant hardware. The title is concise, specific, and accurately conveys the primary change without unnecessary verbosity.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h (1)

109-118: LGTM! Compile-time check correctly expanded for B200 support.

The conditional now properly includes SM10x (SM100-109) devices in addition to SM89 and SM12x, and the error message accurately reflects "Blackwell devices." The logic aligns with the runtime dispatch changes in fp8_rowwise_gemm_template.h.

Optional: Consider renaming the struct.

The struct name Sm89_12xOnly is now slightly misleading since it also supports SM10x devices. A name like Sm89_Blackwell would be more accurate, though this is a low-priority refactor.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bd740c9 and 7750560.

📒 Files selected for processing (5)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt (1 hunks)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp (2 hunks)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h (1 hunks)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h (2 hunks)
  • tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Use only spaces, no tabs; indent with 4 spaces.

Files:

  • tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.py

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.py: Python code must target Python 3.8+.
Indent Python code with 4 spaces; do not use tabs.
Maintain module namespace when importing; prefer 'from package.subpackage import foo' then 'foo.SomeClass()' instead of importing the class directly.
Python filenames should be snake_case (e.g., some_file.py).
Python classes use PascalCase names.
Functions and methods use snake_case names.
Local variables use snake_case; prefix 'k' for variables that start with a number (e.g., k_99th_percentile).
Global variables use upper SNAKE_CASE prefixed with 'G' (e.g., G_MY_GLOBAL).
Constants use upper SNAKE_CASE (e.g., MY_CONSTANT).
Avoid shadowing variables from an outer scope.
Initialize all externally visible members of a class in the constructor.
Prefer docstrings for interfaces that may be used outside a file; comments for in-function or file-local interfaces.
Use Google-style docstrings for classes and functions (Sphinx-parsable).
Document attributes and variables inline so they render under the class/function docstring.
Avoid reflection when a simpler, explicit approach suffices (e.g., avoid dict(**locals()) patterns).
In try/except, catch the most specific exceptions possible.
For duck-typing try/except, keep the try body minimal and use else for the main logic.

Files:

  • tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py
**/*.{cpp,cxx,cc,h,hpp,hh,hxx,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Prepend the NVIDIA Apache-2.0 copyright header with current year to the top of all source files (e.g., .cpp, .h, .cu, .py).

Files:

  • tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh}: Namespace closing braces must include a trailing comment with the namespace name (e.g., '} // namespace foo').
Prefer const or constexpr variables over #define for constants.
Declare variables that are not modified after initialization as const.
Avoid magic literals in code; except for 0, nullptr, true, false. Use named constants for comparisons and logic.
Use Allman brace style for formatting.
Place the semicolon of an empty for/while loop on a new line.
Bodies of switch/while/do-while/for must be compound statements (brace-delimited), and if/else must always be followed by brace-delimited statements.
Type names (e.g., classes) must be CamelCase starting with an uppercase letter (e.g., FooBar).
Local variables, methods, and namespaces use lowerCamelCase (e.g., localFooBar).
Non-magic-number global variables that are non-static and not in an anonymous namespace must be lowerCamelCase prefixed with 'g' (e.g., gDontUseGlobalFoos).
Non-magic-number globals that are static or in an anonymous namespace use lowerCamelCase prefixed with 's' (e.g., sMutableStaticGlobal).
Locally visible static variables use lowerCamelCase with 's' prefix (e.g., static std::once_flag sFlag).
Private/protected member variables use 'm' prefix with CamelCase (e.g., mNbFooValues). Public members may omit, but 'm' is encouraged for clarity.
Constants (enums, global constants, static constants, and function-scope magic/literal constants) use uppercase SNAKE_CASE with 'k' prefix (e.g., kDIGIT_NUM).
Function-scope constants that are not magic numbers or literals are named like non-constant variables (e.g., bool const pass = a && b).
If macros are necessary, name them in UPPER_SNAKE_CASE (e.g., FOO_VERSION) and prefer constants over #define.
Use LLVM clang-format; wrap lines at a maximum of 120 columns; use '// clang-format off/on' sparingly with justification.
Use smart pointers for heap allocations; prefer unique_ptr for sole ownership, shared_ptr for shared...

Files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.{cpp,cxx,cc,cu,h,hpp,hh,hxx,cuh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

C++ filenames should be lowerCamelCase (first letter lowercase) and must be case-insensitive unique within a compilation target.

Files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.{h,hpp,hh,hxx}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Document new class interfaces and function prototypes with Doxygen; use //! for single-line and //!< for members.

Files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.{h,hpp,hh,hxx,cpp,cxx,cc}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.{h,hpp,hh,hxx,cpp,cxx,cc}: Prefer anonymous namespaces over 'static' for internal linkage of functions.
All templates (class/function/member/static) must be instantiated at least once; non-POD classes should have private data members.

Files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
**/*.{h,hpp,hh,hxx,cuh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Use include guards named 'TRTLLM_<FILE_NAME_IN_CAPS_WITH_UNDERSCORES>_H' (no leading or trailing underscore; directory names excluded).

Files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
🧬 Code graph analysis (2)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm89.h (3)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/mxfp8_mxfp4_gemm_template_sm100.h (2)
  • tensorrt_llm (44-294)
  • kernels (46-293)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/nvfp4_nvfp4_gemm_template_sm120.h (2)
  • tensorrt_llm (42-262)
  • kernels (44-261)
cpp/include/tensorrt_llm/kernels/archCondition.h (2)
  • tensorrt_llm (19-113)
  • arch (72-111)
cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp (1)
cpp/tests/unit_tests/kernels/mixtureOfExpertsTest.cu (4)
  • sm (1112-1139)
  • sm (1112-1112)
  • sm (1141-1179)
  • sm (1141-1141)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pre-commit Check
🔇 Additional comments (4)
cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt (1)

203-203: LGTM! B200 architecture support added correctly.

The addition of 100f to the CUDA architectures enables FP8 rowwise GEMM compilation for SM100 (B200) devices with forward compatibility.

tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py (1)

3-3: LGTM! Test now enabled for Blackwell devices.

Removing the skip_blackwell import allows this test to run on B200 and other Blackwell devices, providing appropriate test coverage for the new FP8 rowwise GEMM support.

cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h (1)

511-511: Verify SM100+ devices should use SM89 implementation.

The dispatch logic now routes SM100+ devices (including B200 and SM120+) to the SM89 kernel implementation, while SM90 continues using TMA-based kernels. This appears intentional, but please confirm:

  1. SM100 (B200) devices lack TMA-based FP8 rowwise kernels and should use SM89 implementation?
  2. SM120+ devices should also use SM89 implementation rather than dedicated SM120 kernels?

The changes are consistent across both dispatchToArch (Line 511) and getConfigs (Line 578) methods.

Also applies to: 578-578

cpp/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp (1)

178-178: LGTM! Tile configuration heuristics updated for B200 support.

The candidate tile selection for FP8 GEMMs now correctly includes SM100+ devices alongside SM89, ensuring they receive appropriate tile configurations. This change aligns with the dispatch logic updates in fp8_rowwise_gemm_template.h.

Also applies to: 196-196

@achartier achartier marked this pull request as draft October 14, 2025 17:41
@achartier achartier force-pushed the b200_rowwise branch 2 times, most recently from 2775c14 to c4e9b69 Compare October 24, 2025 04:54
@achartier
Copy link
Collaborator Author

@coderabbitai full review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 24, 2025

✅ Actions performed

Full review triggered.

@achartier achartier marked this pull request as ready for review October 24, 2025 05:14
@achartier
Copy link
Collaborator Author

/bot run

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm100.h (1)

95-111: Epilogue types reuse Sm90 fusion on SM100 — confirm CUTLASS compatibility.

Using Sm90* fusion components on SM100 is common but depends on CUTLASS version. Please confirm these aliases remain valid for arch::Sm100 to avoid template selection drift.

If CUTLASS exposes SM100‑specific aliases later, consider switching for clarity.

Also applies to: 121-136

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 59375e8 and d150a0a.

📒 Files selected for processing (4)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt (1 hunks)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm100.h (1 hunks)
  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h (4 hunks)
  • tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-09-23T15:13:48.819Z
Learnt from: nv-lschneider
PR: NVIDIA/TensorRT-LLM#7910
File: cpp/tensorrt_llm/kernels/nccl_device/multimem.h:20-30
Timestamp: 2025-09-23T15:13:48.819Z
Learning: TRT-LLM targets modern CUDA toolkits that support FP8 datatypes, so cuda_fp8.h can be included unconditionally without version guards in TRT-LLM code.

Applied to files:

  • cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h
🪛 Clang (14.0.6)
cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_kernel_template_sm100.h

[error] 24-24: 'cute/tensor.hpp' file not found

(clang-diagnostic-error)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pre-commit Check
🔇 Additional comments (3)
cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt (1)

203-203: Enable SM100 builds for FP8 rowwise GEMM — looks good.

Including 100f on fb_gemm_src matches the new SM100 path. Please verify CI covers 100-real and that CUTLASS for 100 is available in the toolchain.

tests/unittest/_torch/thop/parallel/test_fp8_rowwise_linear.py (1)

3-3: Removing Blackwell skip is appropriate for SM100 enablement.

Confirm CI runs this test on SM100/SM120 runners; keep skip_pre_hopper to avoid pre‑Hopper devices.

cpp/tensorrt_llm/kernels/cutlass_kernels/fp8_rowwise_gemm/fp8_rowwise_gemm_template.h (1)

41-41: SM100 FP8 rowwise GEMM wiring — LGTM with a few checks.

  • New include and SM100 prepare/launcher/dispatch paths mirror SM90 patterns; arguments/strides look consistent.
  • getConfigs adds comprehensive SM100 tiles and cluster shapes; routing mSm==100 is correct.

Please verify:

  • Kernel schedules AUTO work well on SM100 with these tiles; no missing shared‑mem opt‑in paths for large tiles.
  • Ktile constants (128, 256)/sizeof(T) match CUTLASS expectations for FP8 rowwise on SM100.
  • CI covers workspace‑size path (A/B/D null) for at least one config per cluster shape.

Also applies to: 487-519, 521-549, 551-589, 591-662, 689-693, 761-790

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22377 [ run ] triggered by Bot. Commit: d150a0a

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22378 [ run ] triggered by Bot. Commit: d150a0a

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22377 [ run ] completed with state ABORTED. Commit: d150a0a

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22378 [ run ] completed with state SUCCESS. Commit: d150a0a
/LLM/main/L0_MergeRequest_PR pipeline #16868 completed with status: 'FAILURE'

@achartier
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22454 [ run ] triggered by Bot. Commit: a7cc3dd

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22454 [ run ] completed with state SUCCESS. Commit: a7cc3dd
/LLM/main/L0_MergeRequest_PR pipeline #16922 completed with status: 'FAILURE'

@achartier
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22468 [ run ] triggered by Bot. Commit: a7cc3dd

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22468 [ run ] completed with state SUCCESS. Commit: a7cc3dd
/LLM/main/L0_MergeRequest_PR pipeline #16934 completed with status: 'SUCCESS'

Signed-off-by: Aurelien Chartier <[email protected]>
Signed-off-by: Aurelien Chartier <[email protected]>
Signed-off-by: Aurelien Chartier <[email protected]>
Signed-off-by: Aurelien Chartier <[email protected]>
Signed-off-by: Aurelien Chartier <[email protected]>
@achartier
Copy link
Collaborator Author

/bot skip --comment "CI passed earlier"

@achartier achartier enabled auto-merge (squash) October 27, 2025 19:57
@tensorrt-cicd
Copy link
Collaborator

PR_Github #22677 [ skip ] triggered by Bot. Commit: 5e8a304

@tensorrt-cicd
Copy link
Collaborator

PR_Github #22677 [ skip ] completed with state SUCCESS. Commit: 5e8a304
Skipping testing for commit 5e8a304

@achartier achartier merged commit 1401a3c into NVIDIA:main Oct 27, 2025
5 checks passed
@achartier achartier deleted the b200_rowwise branch October 27, 2025 22:49
symphonylyh pushed a commit to symphonylyh/TensorRT-LLM that referenced this pull request Oct 29, 2025
@achartier
Copy link
Collaborator Author

@coderabbitai configuration

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 30, 2025

CodeRabbit configuration
language: en-US
tone_instructions: ''
early_access: false
enable_free_tier: true
reviews:
  profile: chill
  request_changes_workflow: false
  high_level_summary: true
  high_level_summary_placeholder: '@coderabbitai summary'
  high_level_summary_in_walkthrough: false
  auto_title_placeholder: '@coderabbitai title'
  auto_title_instructions: >-
    Format: "[<category>] <title>". Category must be one of: fix, feat, doc,
    infra, style, refactor, perf, test, chore, revert. Enclose the category in
    square brackets. Title should be concise (<= 60 chars). Example: "[feat] Add
    logit_bias support".
  review_status: false
  commit_status: false
  fail_commit_status: false
  collapse_walkthrough: true
  changed_files_summary: true
  sequence_diagrams: true
  estimate_code_review_effort: true
  assess_linked_issues: true
  related_issues: true
  related_prs: true
  suggested_labels: true
  labeling_instructions: []
  auto_apply_labels: false
  suggested_reviewers: true
  auto_assign_reviewers: false
  in_progress_fortune: true
  poem: false
  path_filters: []
  path_instructions: []
  abort_on_close: true
  disable_cache: false
  auto_review:
    enabled: true
    auto_incremental_review: false
    ignore_title_keywords: []
    labels: []
    drafts: false
    base_branches:
      - main
      - release/.+
    ignore_usernames: []
  finishing_touches:
    docstrings:
      enabled: true
    unit_tests:
      enabled: true
  pre_merge_checks:
    docstrings:
      mode: warning
      threshold: 80
    title:
      mode: warning
      requirements: ''
    description:
      mode: warning
    issue_assessment:
      mode: warning
    custom_checks: []
  tools:
    ast-grep:
      rule_dirs: []
      util_dirs: []
      essential_rules: true
      packages: []
    shellcheck:
      enabled: true
    ruff:
      enabled: true
    markdownlint:
      enabled: true
    github-checks:
      enabled: true
      timeout_ms: 90000
    languagetool:
      enabled: true
      enabled_rules: []
      disabled_rules: []
      enabled_categories: []
      disabled_categories: []
      enabled_only: false
      level: default
    biome:
      enabled: true
    hadolint:
      enabled: true
    swiftlint:
      enabled: true
    phpstan:
      enabled: true
      level: default
    phpmd:
      enabled: true
    phpcs:
      enabled: true
    golangci-lint:
      enabled: true
    yamllint:
      enabled: true
    gitleaks:
      enabled: true
    checkov:
      enabled: true
    detekt:
      enabled: true
    eslint:
      enabled: true
    flake8:
      enabled: true
    rubocop:
      enabled: true
    buf:
      enabled: true
    regal:
      enabled: true
    actionlint:
      enabled: true
    pmd:
      enabled: true
    clang:
      enabled: true
    cppcheck:
      enabled: true
    semgrep:
      enabled: true
    circleci:
      enabled: true
    clippy:
      enabled: true
    sqlfluff:
      enabled: true
    prismaLint:
      enabled: true
    pylint:
      enabled: true
    oxc:
      enabled: true
    shopifyThemeCheck:
      enabled: true
    luacheck:
      enabled: true
    brakeman:
      enabled: true
    dotenvLint:
      enabled: true
    htmlhint:
      enabled: true
    checkmake:
      enabled: true
    osvScanner:
      enabled: true
chat:
  art: true
  auto_reply: true
  integrations:
    jira:
      usage: auto
    linear:
      usage: auto
knowledge_base:
  opt_out: false
  web_search:
    enabled: true
  code_guidelines:
    enabled: true
    filePatterns:
      - '**/CODING_GUIDELINES.md'
  learnings:
    scope: auto
  issues:
    scope: auto
  jira:
    usage: auto
    project_keys: []
  linear:
    usage: auto
    team_keys: []
  pull_requests:
    scope: auto
  mcp:
    usage: auto
    disabled_servers: []
code_generation:
  docstrings:
    language: en-US
    path_instructions: []
  unit_tests:
    path_instructions: []

dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 1, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants