Skip to content

Conversation

@QiJune
Copy link
Collaborator

@QiJune QiJune commented Oct 17, 2025

Summary by CodeRabbit

  • Tests
    • Updated test configurations for A10 GPU environments with adjusted resource allocation settings.
    • Expanded integration test suite coverage for disaggregated inference scenarios and end-to-end workflows on A10 GPUs.
    • Adjusted test matrix for H100 GPU environments by refining included test scenarios.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@QiJune QiJune requested review from a team as code owners October 17, 2025 05:28
@QiJune QiJune requested review from kxdc and poweiw October 17, 2025 05:28
@QiJune
Copy link
Collaborator Author

QiJune commented Oct 17, 2025

/bot run --stage-list "A10-PyTorch-1,A10-PyTorch-2"

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 17, 2025

📝 Walkthrough

Walkthrough

The changes update test configurations across three files: modifying GPU and split allocations in the L0 test configuration, adding disaggregated and end-to-end tests to the A10 test matrix, and removing tests from the H100 test matrix.

Changes

Cohort / File(s) Summary
Groovy Test Configuration
jenkins/L0_Test.groovy
Modified A10-PyTorch-1 entry to use 2 splits instead of 1, and added new A10-PyTorch-2 entry with 2 GPUs across 2 splits.
A10 Test Matrix
tests/integration/test_lists/test-db/l0_a10.yml
Added 20+ new test entries under PyTorch pre-merge section, including disaggregated utility tests, router tests, cluster manager tests, disaggregated integration tests with TinyLlama model variants, and OpenAI e2e tests.
H100 Test Matrix
tests/integration/test_lists/test-db/l0_h100.yml
Removed multiple disaggregated tests, disaggregated integration tests, and e2e tests from the H100 test matrix across pre-merge and post-merge blocks.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The PR description is largely incomplete and does not follow the repository's description template. The Description section is entirely empty with only the template comment remaining, and the Test Coverage section is similarly unfilled. While the PR Checklist is present, only the final checkbox is marked without substantive content in the preceding sections that explain what is being changed and why. The description provides no explanation of the motivation, scope, or test coverage for the test redistribution between H100 and A10 configurations.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The PR title "[None][ci] move some test cases from H100 to A10" clearly and directly relates to the changeset. The raw summary shows removals from l0_h100.yml, additions to l0_a10.yml, and updates to jenkins/L0_Test.groovy to support A10-PyTorch-2, which aligns precisely with the title's description of moving tests between GPU types. The title is specific, concise, and avoids vague terminology, providing sufficient context for a reviewer scanning commit history to immediately understand the primary change.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1e1f430 and a63354f.

📒 Files selected for processing (3)
  • jenkins/L0_Test.groovy (1 hunks)
  • tests/integration/test_lists/test-db/l0_a10.yml (1 hunks)
  • tests/integration/test_lists/test-db/l0_h100.yml (0 hunks)
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/test-db/l0_h100.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pre-commit Check
🔇 Additional comments (3)
tests/integration/test_lists/test-db/l0_a10.yml (2)

29-55: Verify A10 capacity for expanded test suite.

Significant test additions have been made to the A10 configuration:

  • 6 new disaggregated scenario tests (lines 35-40)
  • 4 new workers tests (lines 41-44)
  • 4 new single_gpu tests with parameter variations (lines 45-48)
  • 2 new OpenAI tests (lines 54-55)

The use of TinyLlama-1.1B-Chat-v1.0 is appropriate for A10 hardware. However, please verify:

  1. A10 GPUs can reliably handle this expanded test suite
  2. The split configuration (2 splits) in jenkins/L0_Test.groovy is sufficient to keep individual job runtime reasonable
  3. These tests were previously passing on H100 and are expected to pass on A10

24-28: Disaggregated tests successfully moved to A10 configuration.

Verification confirms that all five disaggregated unit tests have been removed from H100 and are now present in the A10 configuration at lines 24–28. The test movement is complete and correct.

jenkins/L0_Test.groovy (1)

2435-2436: LGTM! Configuration aligns with expanded test suite.

The split of A10-PyTorch tests from 1 job to 2 parallel jobs is appropriate given the test additions in l0_a10.yml. The configuration syntax is correct:

  • A10-PyTorch-1: Split 1 of 2
  • A10-PyTorch-2: Split 2 of 2

This should help distribute the increased test load and keep individual job runtimes reasonable. The format is consistent with other multi-split configurations in the file (e.g., A10-TensorRT stages with 6 splits).


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21660 [ run ] triggered by Bot

Signed-off-by: junq <[email protected]>
@QiJune
Copy link
Collaborator Author

QiJune commented Oct 17, 2025

/bot run --stage-list "A10-PyTorch-1,A10-PyTorch-2"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21674 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21660 [ run ] completed with state ABORTED
LLM/main/L0_MergeRequest_PR #16321 (Blue Ocean) completed with status: ABORTED

@QiJune QiJune requested a review from chzblych October 17, 2025 08:19
@tensorrt-cicd
Copy link
Collaborator

PR_Github #21674 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #16332 (Partly Tested) completed with status: 'FAILURE'

Signed-off-by: junq <[email protected]>
@QiJune
Copy link
Collaborator Author

QiJune commented Oct 20, 2025

/bot run --gpu-type "A10"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21823 [ run ] triggered by Bot. Commit: cdc3de9

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21823 [ run ] completed with state SUCCESS. Commit: cdc3de9
/LLM/main/L0_MergeRequest_PR pipeline #16448 (Partly Tested) completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

@QiJune
Copy link
Collaborator Author

QiJune commented Oct 20, 2025

/bot skip --comment "all A10 stages passed"

@QiJune QiJune enabled auto-merge (squash) October 20, 2025 05:31
@tensorrt-cicd
Copy link
Collaborator

PR_Github #21855 [ skip ] triggered by Bot. Commit: cdc3de9

@tensorrt-cicd
Copy link
Collaborator

PR_Github #21855 [ skip ] completed with state SUCCESS. Commit: cdc3de9
Skipping testing for commit cdc3de9

@QiJune QiJune merged commit d05079b into NVIDIA:main Oct 20, 2025
5 checks passed
govind-ramnarayan pushed a commit to nv-auto-deploy/TensorRT-LLM that referenced this pull request Oct 21, 2025
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request Oct 24, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 1, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants