Fully deprecate AutoGPTQ for GPT-QModel by ZX-ModelCloud · Pull Request #2385 · huggingface/optimum

ZX-ModelCloud · 2025-11-28T02:20:39Z

What does this PR do?

Remove autogptq clutter and autogptq related configs that are not worth adding backward compat.
See
transformers PR#41567
peft PR#2932

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

Who can review?

Signed-off-by: ZX-ModelCloud <[email protected]>

…nsformers. Signed-off-by: ZX-ModelCloud <[email protected]>

Signed-off-by: ZX-ModelCloud <[email protected]>

…arguments Signed-off-by: ZX-ModelCloud <[email protected]>

Qubitium · 2025-11-28T06:34:00Z

@SunMarc @BenjaminBossan This is the companion PR to both PEFT PR huggingface/peft#2917 and Transformer PR huggingface/transformers#41567.

Also fixes a Transformer bug (latest changes broke Optimum) in 14d5cfe

Major changes is kernel selection to gptqmodel is unified under new hf_select_quant_linear_v2 api for both gptq and pending awq.

Related Optimum quantization CI tests passing.

Signed-off-by: ZX-ModelCloud <[email protected]>

SunMarc

Thanks for updating the codebase and deprecating auto-gptq in optimum. Note that in the long term, we will probably not maintain this anymore and we should redirect users to appropriate lib like gptq-model. Feel free to update the docs to prompt users to direct use gptq-model.
In transformers, we only deal with quantizing ourselves if we can do it on the fly without calibration. gptq was one of the first method, so this is why it was supported through optimum but for example, we didn't do it for awq.

Signed-off-by: ZX-ModelCloud <[email protected]>

Qubitium · 2025-12-02T09:10:56Z

@IlyasMoutawwakil @SunMarc PR is now synced to Peft/Transformer pending Prs. Ready for final review for this portion. All relevant tests passing.

Signed-off-by: ZX-ModelCloud <[email protected]>

IlyasMoutawwakil · 2025-12-02T10:04:44Z

Thanks ! can you please add GPTQModel to the gptq workflow:
https://github.com/huggingface/optimum/blob/main/.github/workflows/test_gptq.yml#L45
and run code styling in a clean env

Signed-off-by: ZX-ModelCloud <[email protected]>

IlyasMoutawwakil · 2025-12-02T12:29:03Z

 def load_quantized_model(
    model: nn.Module,
    save_folder: str,
+    backend: BACKEND = BACKEND.AUTO,


BACKEND is only defined when GPTQModel is available which breaks this file when it isn't

HuggingFaceDocBuilderDev · 2025-12-02T13:12:19Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Signed-off-by: ZX-ModelCloud <[email protected]>

Qubitium · 2025-12-04T13:09:39Z

-                backend=BACKEND.EXLLAMA_V1,
+                backend=BACKEND.EXLLAMA_V2,


@IlyasMoutawwakil Note we change the ci to test exllama v2 kernel since GPT-QModel no longer compile Exallma v1 kernel by default due to a) save on whl size b) v2 is consistently faster so there is no reason to use v1 kernel. exllama v1 kernel is still compilable from gpt-qmodel src but for the sake or fast optimum ci (install from precompiled whl), we are moving away from exallma v1 tests.

thanks for taking care of this !

IlyasMoutawwakil · 2025-12-17T09:09:50Z

+      - name: Install dependencies
+        run: |
+          pip install --upgrade pip uv
+          uv pip install .[tests]
+          uv pip install gptqmodel --no-build-isolation


@Qubitium @ZX-ModelCloud is there anything that needs to be fixed in the installation process her ? it fails with :

× Failed to download and build `gptqmodel==5.4.2` ╰─▶ Package metadata version `5.4.2+cu128torch2.9` does not match `5.4.2` from the wheel filename

@Qubitium @ZX-ModelCloud is there anything that needs to be fixed in the installation process her ? it fails with :

× Failed to download and build `gptqmodel==5.4.2` ╰─▶ Package metadata version `5.4.2+cu128torch2.9` does not match `5.4.2` from the wheel filename

Please pin ci test to gptqmodel >= 5.6.10

There is a bug in uv. We previously submitted and they fixed this meta match issue but now regressing in latest uv. Please use pin gptqmodel to 5.6.10 and avoid uv

i pinned gptqmodel to 5.6.10, let's see if it works,
avoiding uv is kinda hard once you get used to its speedup gains 😅

i pinned gptqmodel to 5.6.10, let's see if it works, avoiding uv is kinda hard once you get used to its speedup gains 😅

We will try to get this resolved with uv upstream (if we cannot resolve it on our end with setup.py hacks). Need to stick with pip for gptqmodel install for now. =(

it worked now !
so gptqmodel needs to build the cuda/hip/sycl kernels from scratch at installation time ? did you consider using the kernels and kernel-builder libs ? would make the package more universal and solve most distribution issues.

Qubitium · 2025-12-17T10:07:11Z

@IlyasMoutawwakil We may have an internal issue.

pip install "gptqmodel>=5.6.10" --no-build-isolation

also failing with same meta data parsing error...checking

IlyasMoutawwakil

LGTM ! thanks for simplifying this !

Qubitium · 2025-12-17T11:53:03Z

@IlyasMoutawwakil uv re-enabled. We fixed uv compat. 5.6.12 releasd to pypi and prebuilt wheels are being created right now (likely take 1 hour to complete all the wheels). https://github.com/ModelCloud/GPTQModel/actions/runs/20301283813/job/58308710953

Once the wheels are done, you can trigger ci, and uv should install download the prebuilt wheel without having to build the full kernels.

Co-authored-by: Ilyas Moutawwakil <[email protected]>

Remove autogptq clutter and autogptq related configs that are not worth adding backward compat. For reference, see - huggingface/transformers#41567 - huggingface/optimum#2385 * fix gptq test Signed-off-by: ZX-ModelCloud <[email protected]> * remove auto_gptq Signed-off-by: ZX-ModelCloud <[email protected]> * call hf_select_quant_linear_v2() Signed-off-by: ZX-ModelCloud <[email protected]> * remove auto_awq Signed-off-by: ZX-ModelCloud <[email protected]> * cleanup Signed-off-by: ZX-ModelCloud <[email protected]> * format Signed-off-by: ZX-ModelCloud <[email protected]> * fix PeftAwqGPUTests Signed-off-by: ZX-ModelCloud <[email protected]> * update GPTQMODEL_MINIMUM_VERSION to 5.6.0 Signed-off-by: ZX-ModelCloud <[email protected]> * update GPTQMODEL_MINIMUM_VERSION Signed-off-by: ZX-ModelCloud <[email protected]> * Update minimum version requirement for gptqmodel * style * call is_gptqmodel_available() Signed-off-by: ZX-ModelCloud <[email protected]> * call is_gptqmodel_available() Signed-off-by: ZX-ModelCloud <[email protected]> * Update minimum version for gptqmodel to 5.6.12 * When hf_device_map does not exist, infer the device_map Signed-off-by: ZX-ModelCloud <[email protected]> * cleanup Signed-off-by: ZX-ModelCloud <[email protected]> --------- Signed-off-by: ZX-ModelCloud <[email protected]> Co-authored-by: Qubitium-ModelCloud <[email protected]> Co-authored-by: githubnemo <[email protected]>

ZX-ModelCloud added 6 commits November 26, 2025 02:57

call hf_select_quant_linear_v2()

b107472

Signed-off-by: ZX-ModelCloud <[email protected]>

Remove the import of auto_gptq

fb566a0

Signed-off-by: ZX-ModelCloud <[email protected]>

Remove the import of auto_gptq in unittests

f1d0030

Signed-off-by: ZX-ModelCloud <[email protected]>

Fixed the incorrect layer_output when using the latest version of tra…

14d5cfe

…nsformers. Signed-off-by: ZX-ModelCloud <[email protected]>

use assertEqual instead of assertTrue

272d1e4

Signed-off-by: ZX-ModelCloud <[email protected]>

load_quantized_model() remove "disable_exllama" and "exllama_config" …

40d8a81

…arguments Signed-off-by: ZX-ModelCloud <[email protected]>

ZX-ModelCloud added 3 commits November 28, 2025 06:55

fix gptq unittest

7a9612a

Signed-off-by: ZX-ModelCloud <[email protected]>

fix gptq unittest

8ffac92

Signed-off-by: ZX-ModelCloud <[email protected]>

setting offload_to_disk=False

e6da59e

Signed-off-by: ZX-ModelCloud <[email protected]>

SunMarc approved these changes Nov 28, 2025

View reviewed changes

SunMarc requested a review from IlyasMoutawwakil November 28, 2025 10:39

ZX-ModelCloud mentioned this pull request Dec 1, 2025

Fully deprecate AutoGPTQ and AutoAWQ for GPT-QModel huggingface/peft#2932

Merged

remove auto_gptq

fd3f211

Signed-off-by: ZX-ModelCloud <[email protected]>

ZX-ModelCloud changed the title ~~[WIP] Fully deprecate AutoGPTQ for GPT-QModel~~ Fully deprecate AutoGPTQ for GPT-QModel Dec 2, 2025

ZX-ModelCloud marked this pull request as ready for review December 2, 2025 09:08

format

9f8e92b

Signed-off-by: ZX-ModelCloud <[email protected]>

format

237d323

Signed-off-by: ZX-ModelCloud <[email protected]>

IlyasMoutawwakil reviewed Dec 2, 2025

View reviewed changes

IlyasMoutawwakil added 2 commits December 2, 2025 13:43

fix

c5e6115

fix

2b7449a

remove ExllamaV1 Test

fbd4ed7

Signed-off-by: ZX-ModelCloud <[email protected]>

Qubitium reviewed Dec 4, 2025

View reviewed changes

IlyasMoutawwakil and others added 2 commits December 5, 2025 12:25

update ci

30455cf

Merge branch 'huggingface:main' into compat-gptqmodel

c334f96

IlyasMoutawwakil reviewed Dec 17, 2025

View reviewed changes

test

39f270f

avoid uv

439043d

IlyasMoutawwakil approved these changes Dec 17, 2025

View reviewed changes

re-enable uv and pin to >= 5.6.12

10b8479

IlyasMoutawwakil reviewed Dec 17, 2025

View reviewed changes

Comment thread .github/workflows/test_gptq.yml Outdated

Update .github/workflows/test_gptq.yml

5e73030

Co-authored-by: Ilyas Moutawwakil <[email protected]>

IlyasMoutawwakil merged commit 3a85d51 into huggingface:main Dec 17, 2025
16 checks passed

Qubitium deleted the compat-gptqmodel branch December 17, 2025 15:14

Uh oh!

Conversation

ZX-ModelCloud commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

Qubitium commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SunMarc left a comment

Choose a reason for hiding this comment

Uh oh!

Qubitium commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

IlyasMoutawwakil commented Dec 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Dec 2, 2025

Uh oh!

Qubitium Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Qubitium commented Dec 17, 2025

Uh oh!

IlyasMoutawwakil left a comment

Choose a reason for hiding this comment

Uh oh!

Qubitium commented Dec 17, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ZX-ModelCloud commented Nov 28, 2025 •

edited

Loading

Qubitium commented Nov 28, 2025 •

edited

Loading

Qubitium commented Dec 2, 2025 •

edited

Loading

Qubitium Dec 4, 2025 •

edited

Loading