Skip to content

Add TRT 70B (FP8 and FP4)#2

Closed
kedarpotdar-nv wants to merge 42 commits intomainfrom
fp4-init
Closed

Add TRT 70B (FP8 and FP4)#2
kedarpotdar-nv wants to merge 42 commits intomainfrom
fp4-init

Conversation

@kedarpotdar-nv
Copy link
Copy Markdown
Collaborator

following up from #1.

Made these changes:

  1. added 70B TRT LLM config to 70b-tmpl.yml
  2. added new var for precision - fp8 or fp4
  3. ensured collect and plot scripts are reflective of trt and vllm

See workflow run # 134 which had sanity test.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant