[CORE] Add vLLM Backend for FORMAT.GPTQ #190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

Qubitium merged 17 commits into ModelCloud:main from PZS-ModelCloud:add_vlm_sglang

Jul 10, 2024

Contributor

PZS-ModelCloud commented Jul 9, 2024

No description provided.

PZS-ModelCloud added 5 commits

July 9, 2024 08:57


          add vllm load support

67b5373


          add sglang

b7af90e


          fix vllm load model show kv_caches error

fcf0d22


          revert sglang

216941d


          mod clean up

51323ad

Qubitium reviewed

View reviewed changes

gptqmodel/models/base.py Outdated Show resolved Hide resolved

Qubitium reviewed

View reviewed changes

gptqmodel/models/base.py Outdated Show resolved Hide resolved


          Update base.py

ee9ac00

Qubitium reviewed

View reviewed changes

gptqmodel/models/base.py Outdated Show resolved Hide resolved

Qubitium reviewed

View reviewed changes

tests/test_vllm.py Outdated Show resolved Hide resolved

Qubitium reviewed

View reviewed changes

gptqmodel/utils/vllm.py Outdated Show resolved Hide resolved

Qubitium reviewed

View reviewed changes

tests/test_vllm.py Outdated Show resolved Hide resolved

Qubitium reviewed

View reviewed changes

gptqmodel/models/base.py Outdated Show resolved Hide resolved

Qubitium changed the title ~~Add vlm~~ [CORE]Add vLLM Backend for FORMAT.GPTQ

Qubitium changed the title ~~[CORE]Add vLLM Backend for FORMAT.GPTQ~~ [CORE] Add vLLM Backend for FORMAT.GPTQ

Qubitium and others added 11 commits

July 10, 2024 02:25


          Update base.py

a93219a


          Update base.py

1a5dafd


          Update test_vllm.py

9001c3d


          Update vllm.py

db1dcc8


          Update base.py

4e7ffb5


          Update vllm.py

40f0788


          Merge branch 'ModelCloud:main' into add_vlm_sglang

848c68c


          add convert_hf_params_to_vllm and clean up

30584bd


          format code

74bbedb


          mod clean up

ab232c1


          mod clean up

b736478

Qubitium merged commit 3951416 into ModelCloud:main

PZS-ModelCloud deleted the add_vlm_sglang branch

July 10, 2024 03:08

DeJoker pushed a commit to DeJoker/GPTQModel that referenced this pull request


          [CORE] Add vLLM Backend for FORMAT.GPTQ (ModelCloud#190)

40308cd

* add vllm load support

* add sglang

* fix vllm load model show kv_caches error

* revert sglang

* mod clean up

* Update base.py

* Update base.py

* Update base.py

* Update test_vllm.py

* Update vllm.py

* Update base.py

* Update vllm.py

* add convert_hf_params_to_vllm and clean up

* format code

* mod clean up

* mod clean up

---------

Co-authored-by: Qubitium-ModelCloud <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet