-
Notifications
You must be signed in to change notification settings - Fork 34k
[PoC] HF exporters #41992
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
[PoC] HF exporters #41992
Changes from all commits
Commits
Show all changes
406 commits
Select commit
Hold shift + click to select a range
656161b
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 0ad9382
style
IlyasMoutawwakil eb1712b
revert deepseek
IlyasMoutawwakil 67a4d30
style
IlyasMoutawwakil 85de2da
fix style
IlyasMoutawwakil 4c94907
fix lfm cache
IlyasMoutawwakil 47fb0a5
revert idefics
IlyasMoutawwakil 59044fd
revert idefics
IlyasMoutawwakil 0886f38
revert deepseek vl
IlyasMoutawwakil 24d4735
better leaf tensors handling
IlyasMoutawwakil 4eee818
revert patchtsmixer
IlyasMoutawwakil 7c87d35
fix in onnx exporter order and executorch backend preparation
IlyasMoutawwakil a9bed19
Merge branch 'main' into hf-exporters
IlyasMoutawwakil b9a547d
fix get_rope_index models and chunked vision attn models
IlyasMoutawwakil a2d1e8c
fix annotation
IlyasMoutawwakil b6d68e5
skip video llama 3
IlyasMoutawwakil 17156c0
skip ernie vl as well
IlyasMoutawwakil 969dbc3
introduce dynamo patches and make prepare_for_export the least impact…
IlyasMoutawwakil eaabee6
Merge branch 'main' into hf-exporters
IlyasMoutawwakil b21385e
Merge branch 'hf-exporters' of https://github.com/huggingface/transfo…
IlyasMoutawwakil b75ce43
Merge branch 'main' into hf-exporters
IlyasMoutawwakil ff50ce8
fix after merge
IlyasMoutawwakil 1d81213
ultimatevlm fix (precompute stuff)
IlyasMoutawwakil 2fe1d11
fix vlm encoders export through optional precomputed inputs
IlyasMoutawwakil 16cbe4d
more vlms passing (t5 gemma and paddle ocr)
IlyasMoutawwakil 96e74bc
fix qwen omni and paddle ocr export
IlyasMoutawwakil 4a1303e
fix docs
IlyasMoutawwakil e814592
fix torch dep
IlyasMoutawwakil a6bcb66
annotations
IlyasMoutawwakil a955ae5
revert change
IlyasMoutawwakil 3bf6a90
missing licence
IlyasMoutawwakil 35b4db7
attempt fix styling
IlyasMoutawwakil 753e70e
Merge branch 'main' into hf-exporters
IlyasMoutawwakil c508d22
fix torch leak
IlyasMoutawwakil 6a087a5
fix
IlyasMoutawwakil 0a196a2
style
IlyasMoutawwakil 2b928ae
fix
IlyasMoutawwakil ee8c7b0
style
IlyasMoutawwakil c3a0f24
guard
IlyasMoutawwakil 87b2ca8
same name for torch patches
IlyasMoutawwakil f024c9d
fix detectron failure
IlyasMoutawwakil adf90da
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 16720f9
fix musicflamingo
IlyasMoutawwakil 4ad2a99
fix
IlyasMoutawwakil 1ac2de8
fix
IlyasMoutawwakil 770cc2e
support and fix gemma4
IlyasMoutawwakil 5072edb
update docs
IlyasMoutawwakil 5ac0267
update docs
IlyasMoutawwakil 27ccc0f
make qwen omni audio encoders exportable
IlyasMoutawwakil 5364d98
docs
IlyasMoutawwakil 0a7baff
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 9bcca9d
get cu seqlens
IlyasMoutawwakil ca6cfdb
leave paddle ocr
IlyasMoutawwakil d52bd2e
skip paddle ocr
IlyasMoutawwakil efafb3c
fix
IlyasMoutawwakil 70862ce
fix layoutlmv2 onnx export
IlyasMoutawwakil 0cc6e42
fix idefics3 on cpu
IlyasMoutawwakil ba8f730
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 7f9382c
fix falcon_mamba and other idefics
IlyasMoutawwakil e2e951d
Merge branch 'main' into hf-exporters
IlyasMoutawwakil e44c5ae
style and docs
IlyasMoutawwakil 1a7e790
dict decomposition
IlyasMoutawwakil 677a90d
remove deferred
IlyasMoutawwakil dab2d14
executorch fixes
IlyasMoutawwakil 6ec03ba
dynamo doctests passing *-*
IlyasMoutawwakil 5eba3d5
clean title
IlyasMoutawwakil c5c7020
update
IlyasMoutawwakil 679eb1b
normalize qwen omni
IlyasMoutawwakil ff66d95
only pure functions
IlyasMoutawwakil f008002
patch views for exeuctorch
IlyasMoutawwakil aef0409
better vision dynamic tensors pre computation
IlyasMoutawwakil 2ae78c7
audio functions
IlyasMoutawwakil e3e66c6
Merge branch 'main' into hf-exporters
IlyasMoutawwakil d8432a5
fix
IlyasMoutawwakil 234f7eb
fix
IlyasMoutawwakil aa220b7
define vision modeling utils
IlyasMoutawwakil a6f9d7b
fix docs
IlyasMoutawwakil 745e36a
style
IlyasMoutawwakil 9eba545
Merge remote-tracking branch 'origin/main' into hf-exporters
IlyasMoutawwakil 41c864e
fix repo
IlyasMoutawwakil 8a5218a
update skips
IlyasMoutawwakil f6f4735
fixes and skips
IlyasMoutawwakil 719157b
fix repo
IlyasMoutawwakil f7e2b06
exportable/compilable paddleocr vl
IlyasMoutawwakil 75977b6
onnx parity
IlyasMoutawwakil 44c93cf
more docs
IlyasMoutawwakil 6114069
style
IlyasMoutawwakil eb0d6d6
move
IlyasMoutawwakil b37d192
avoid lens
IlyasMoutawwakil 98445c3
timesfm minicpmv46 canine colqwen2
IlyasMoutawwakil 32e0ff5
fix test
IlyasMoutawwakil e7dd4aa
executorch fixes
IlyasMoutawwakil e3fdc32
executorch fixes
IlyasMoutawwakil bc2427d
more executorch fixes
IlyasMoutawwakil 139d0e4
Merge branch 'main' into hf-exporters
IlyasMoutawwakil e66362e
Merge branch 'hf-exporters' of https://github.com/huggingface/transfo…
IlyasMoutawwakil a9bacd9
more executorch patches
IlyasMoutawwakil 0c2ff1d
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 8ddd01d
Merge branch 'hf-exporters' of https://github.com/huggingface/transfo…
IlyasMoutawwakil 26f370d
fix
IlyasMoutawwakil 2be769b
Merge branch 'main' into hf-exporters
IlyasMoutawwakil c136f1d
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 25708e0
more fixes
IlyasMoutawwakil 020d874
Update src/transformers/exporters/exporter_dynamo.py
IlyasMoutawwakil 1e975eb
stadarize the patching and fixing api more
IlyasMoutawwakil 93bd416
fix get_encoder is some models
IlyasMoutawwakil ca64a66
Merge branch 'main' into hf-exporters
IlyasMoutawwakil d63f025
fix style
IlyasMoutawwakil 86e3ae9
fix qwen omni get decoder and export
IlyasMoutawwakil 5619ea6
fix xlstm export
IlyasMoutawwakil 01cf3f8
per model type inputs preparation registery
IlyasMoutawwakil f0256c7
standarize torch and library (onnx, executorch) patching and fx progr…
IlyasMoutawwakil 8b604d4
style
IlyasMoutawwakil 104c379
fix gemma4 unified test
IlyasMoutawwakil 4df0ac6
add export for generation pipeline
IlyasMoutawwakil cca77d2
decompose for generation helper function
IlyasMoutawwakil 7537aef
style
IlyasMoutawwakil 3226c29
onnx fixes
IlyasMoutawwakil 3bd85f1
two pat fix
IlyasMoutawwakil 14fbffd
vectorized swin attention
IlyasMoutawwakil 4d6c469
vectorized big bird block sparse attn
IlyasMoutawwakil 4e16cf3
avoid dino timeouts
IlyasMoutawwakil 07f4b68
big bird and pegasus
IlyasMoutawwakil 436ed74
fixed unified gemma4
IlyasMoutawwakil 03a76d3
sam fixes and simplifications
IlyasMoutawwakil 7f6601d
fix repo
IlyasMoutawwakil 0ed56b2
fix style
IlyasMoutawwakil ff54211
unify model instance patches with the rest of the patching api
IlyasMoutawwakil 7925665
change in api to allow exporting multiple models with diff configs
IlyasMoutawwakil 41dbec5
accept dicts
IlyasMoutawwakil 67985c5
docs review comments
IlyasMoutawwakil c13543c
address comments
IlyasMoutawwakil 0390228
fix clvp by not using datasets and vectorizing loops
IlyasMoutawwakil 72eceef
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 05d20b4
style
IlyasMoutawwakil dbdfa97
handle moe per backend and better (e.g. onnx translation)
IlyasMoutawwakil 00857c9
better export workflow, no dict constraint on inputs, less enforcing …
IlyasMoutawwakil c2a0b8e
modeling fixes not honoring config.use_cache
IlyasMoutawwakil 3a17538
style
IlyasMoutawwakil 19ce6fc
style
IlyasMoutawwakil 07d864d
easier to mantain inputs preparation
IlyasMoutawwakil 87081a2
fix chunk on cuda only
IlyasMoutawwakil b9e7919
onnx fixes
IlyasMoutawwakil 962e70b
drop cpu gpu sync in mask update
IlyasMoutawwakil 5bf9b8a
lower onnx export time for some models by disabling optimize
IlyasMoutawwakil 61e9ab8
fix executorch
IlyasMoutawwakil c3ef445
added version drift warnings
IlyasMoutawwakil 26e53fe
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 7a59f83
fix more exeutorch
IlyasMoutawwakil 77c6b5b
remove "legacy" name
IlyasMoutawwakil 5b6993a
model fixes for export compatibility
IlyasMoutawwakil 43326df
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 9e9da66
add minicpmv vision utils
IlyasMoutawwakil 6e6e3ac
can compile
IlyasMoutawwakil 1fce6c3
make ssms fully compilable
IlyasMoutawwakil 7ba1546
more ssms
IlyasMoutawwakil ff33d8b
address new comments
IlyasMoutawwakil a2b3a36
fast ci fixes
IlyasMoutawwakil 3a1971b
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 99b2057
address comments
IlyasMoutawwakil 07bc5c7
fix minimax
IlyasMoutawwakil fadcdf8
fix the last falcon fast ci failure
IlyasMoutawwakil 58f6acb
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 224f016
more comments addressed
IlyasMoutawwakil 1c0de9a
attention
IlyasMoutawwakil cdc3790
reference attn
IlyasMoutawwakil 1ea08b6
cleanup
IlyasMoutawwakil 42a6ca1
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 21a8fd1
use can_return_tuple in patchmixer
IlyasMoutawwakil 4983667
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 74d9b47
proper gqa expalanation
IlyasMoutawwakil a9eef0a
avoid 5D attention
IlyasMoutawwakil 83f8fcb
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 7587ae6
granite export fix
IlyasMoutawwakil b738708
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 17f5e97
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 3ef899e
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 83b39d6
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 7caddb1
address modeling comments
IlyasMoutawwakil 6fddf68
update
IlyasMoutawwakil 6090851
address more comments, unify linear attention and make more models su…
IlyasMoutawwakil a1be3ef
style
IlyasMoutawwakil bf4a9c7
style
IlyasMoutawwakil 95f5ae1
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 950df48
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 77d95f3
modeling changes for attn impl compatibility
IlyasMoutawwakil d8ecd8b
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 6f60ac8
no need for patch
IlyasMoutawwakil 17e3d60
fix
IlyasMoutawwakil 12b284f
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 78e80f6
patch is kenrels available
IlyasMoutawwakil 9e73882
fixes
IlyasMoutawwakil 1ecddd3
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil cecb23e
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil b2f9a31
fix lfm2 conv errors
IlyasMoutawwakil 7fa97a0
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil 36d3c4c
diffllama is exportable now
IlyasMoutawwakil 2044624
enable more models
IlyasMoutawwakil f520f19
address most of anton's review comments
IlyasMoutawwakil 5e7bdee
address cyril's comments
IlyasMoutawwakil 707df61
revert conv rename
IlyasMoutawwakil 32929f6
simpler modular
IlyasMoutawwakil d8f85a6
keep max_batch_size check
IlyasMoutawwakil 25873ea
fixes
IlyasMoutawwakil b7d3137
fix moshi cuda graphs
IlyasMoutawwakil 19ee666
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil dd95a83
fix
IlyasMoutawwakil 5f7f241
added layer type class attr in mixers
IlyasMoutawwakil a14a41c
fix repo
IlyasMoutawwakil d11c066
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil fa7f996
fix
IlyasMoutawwakil 4fc00f1
address comments
IlyasMoutawwakil 753b086
address comments
IlyasMoutawwakil 5bbba7f
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil d7ceca5
Merge branch 'hf-exporters' of https://github.com/huggingface/transfo…
IlyasMoutawwakil 5e5c892
address comments
IlyasMoutawwakil cca5865
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 2f4e263
fix merge
IlyasMoutawwakil 6070f0e
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 1faf03e
claude review findings
IlyasMoutawwakil e6883b5
docs comments
IlyasMoutawwakil 5b6d9f9
address comments
IlyasMoutawwakil 49b7037
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil 90adb3e
style
IlyasMoutawwakil 5d7a918
claude review
IlyasMoutawwakil b52f982
Merge branch 'main' into hf-exporters-models
IlyasMoutawwakil d4e05dd
small fix caught by copilot
IlyasMoutawwakil 3f64fba
Merge branch 'hf-exporters-models' into hf-exporters
IlyasMoutawwakil f339f6f
misc fixes after merge
IlyasMoutawwakil f9501ba
onnx fixes
IlyasMoutawwakil a0a1118
fix cpu sdpa tripping
IlyasMoutawwakil 109d201
only on cpu
IlyasMoutawwakil 04a234c
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 8bee7d0
fix merge
IlyasMoutawwakil 37654c6
fix
IlyasMoutawwakil 2927c74
address comments
IlyasMoutawwakil 4eeaeb3
more comments
IlyasMoutawwakil e2a3fd5
nemotron asr
IlyasMoutawwakil 6e97f8a
fix m3
IlyasMoutawwakil 776d8b6
style
IlyasMoutawwakil c06ef0e
Merge branch 'main' into hf-exporters
IlyasMoutawwakil 5f1692e
add tests for utils
IlyasMoutawwakil 94d8c90
Merge branch 'hf-exporters' of https://github.com/huggingface/transfo…
IlyasMoutawwakil c415efa
utils testing
IlyasMoutawwakil de4a8db
style
IlyasMoutawwakil 82727a5
update deprecation
IlyasMoutawwakil cc027ab
trim tests
IlyasMoutawwakil 72ca863
style
IlyasMoutawwakil cff3450
skips for executorch
IlyasMoutawwakil b40ebe4
Merge branch 'main' into hf-exporters
vasqu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| <!--Copyright 2026 The HuggingFace Team. All rights reserved. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with | ||
| the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on | ||
| an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations under the License. | ||
|
|
||
| ⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be | ||
| rendered properly in your Markdown viewer. | ||
|
|
||
| --> | ||
|
|
||
| # Exporters | ||
|
|
||
| New export backends can be added to Transformers by subclassing [`HfExporter`]. | ||
|
|
||
| <Tip> | ||
|
|
||
| Learn how to use the built-in exporters in the [Exporters](../exporters) guide. | ||
|
|
||
| </Tip> | ||
|
|
||
| ## AutoHfExporter | ||
|
|
||
| [[autodoc]] exporters.auto.AutoHfExporter | ||
|
|
||
| ## AutoExportConfig | ||
|
|
||
| [[autodoc]] exporters.auto.AutoExportConfig | ||
|
|
||
| ## HfExporter | ||
|
|
||
| [[autodoc]] exporters.base.HfExporter | ||
|
|
||
| ## DynamoConfig | ||
|
|
||
| [[autodoc]] exporters.configs.DynamoConfig | ||
|
|
||
| ## OnnxConfig | ||
|
|
||
| [[autodoc]] exporters.configs.OnnxConfig | ||
|
|
||
| ## ExecutorchConfig | ||
|
|
||
| [[autodoc]] exporters.configs.ExecutorchConfig |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Copyright 2026 The HuggingFace Inc. team. All rights reserved. | ||
| # Modifications Copyright (C) 2025, Advanced Micro Devices, Inc. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| from .auto import AutoExportConfig, AutoHfExporter, get_hf_exporter, register_export_config, register_exporter | ||
| from .base import HfExporter | ||
| from .configs import DynamoConfig, ExecutorchConfig, ExportConfigMixin, ExportFormat, OnnxConfig | ||
| from .exporter_dynamo import DynamoExporter | ||
| from .exporter_executorch import ExecutorchExporter | ||
| from .exporter_onnx import OnnxExporter | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| # Copyright 2026 The HuggingFace Inc. team. All rights reserved. | ||
| # Modifications Copyright (C) 2025, Advanced Micro Devices, Inc. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| """Auto exporter factory for HuggingFace exporters.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from ..utils import logging | ||
| from .base import HfExporter | ||
| from .configs import ExportConfigMixin, ExportFormat | ||
| from .exporter_dynamo import DynamoConfig, DynamoExporter | ||
| from .exporter_executorch import ExecutorchConfig, ExecutorchExporter | ||
| from .exporter_onnx import OnnxConfig, OnnxExporter | ||
|
|
||
|
|
||
| AUTO_EXPORTER_MAPPING = { | ||
| "executorch": ExecutorchExporter, | ||
| "dynamo": DynamoExporter, | ||
| "onnx": OnnxExporter, | ||
| } | ||
|
|
||
| AUTO_EXPORT_CONFIG_MAPPING = { | ||
| "executorch": ExecutorchConfig, | ||
| "dynamo": DynamoConfig, | ||
| "onnx": OnnxConfig, | ||
| } | ||
|
|
||
| logger = logging.get_logger(__name__) | ||
|
|
||
|
|
||
| class AutoExportConfig: | ||
|
IlyasMoutawwakil marked this conversation as resolved.
|
||
| """ | ||
| The Auto-HF export config class that takes care of automatically dispatching to the correct | ||
| export config given an export config stored in a dictionary. | ||
| """ | ||
|
|
||
| @classmethod | ||
| def from_dict(cls, export_config_dict: dict): | ||
| export_format = export_config_dict.get("export_format") | ||
|
|
||
| if export_format is None: | ||
| raise ValueError("export_config_dict must contain key 'export_format' set to exporter name") | ||
|
|
||
| # Allow passing an ExportFormat enum value or a plain string | ||
| if isinstance(export_format, ExportFormat): | ||
| name = export_format.value | ||
| else: | ||
| name = export_format | ||
|
|
||
| if name not in AUTO_EXPORT_CONFIG_MAPPING: | ||
| raise ValueError( | ||
| f"Unknown exporter type, got {name} - supported exporters are: {list(AUTO_EXPORT_CONFIG_MAPPING.keys())}" | ||
| ) | ||
|
|
||
| target_cls = AUTO_EXPORT_CONFIG_MAPPING[name] | ||
| return target_cls.from_dict(export_config_dict) | ||
|
|
||
|
|
||
| class AutoHfExporter: | ||
| """ | ||
| The Auto-HF expoerter class that takes care of automatically instantiating to the correct | ||
| `HfExporter` given the `ExportConfig`. | ||
| """ | ||
|
|
||
| @classmethod | ||
| def from_config(cls, export_config: ExportConfigMixin | dict, **kwargs) -> HfExporter: | ||
| # Normalize to a dict so ``supports_export_format`` can act as the single gate. | ||
| export_config_dict = export_config.to_dict() if isinstance(export_config, ExportConfigMixin) else export_config | ||
| if not cls.supports_export_format(export_config_dict): | ||
| raise ValueError( | ||
| f"Unsupported export config: {export_config_dict!r}. " | ||
| f"Registered exporters: {sorted(AUTO_EXPORTER_MAPPING)}." | ||
| ) | ||
|
|
||
| export_format = export_config_dict["export_format"] | ||
| name = export_format.value if isinstance(export_format, ExportFormat) else export_format | ||
| return AUTO_EXPORTER_MAPPING[name](**kwargs) | ||
|
|
||
| @classmethod | ||
| def from_pretrained(cls, pretrained_model_name_or_path, **kwargs) -> HfExporter | None: | ||
| """ | ||
| Load an exporter instance from a pretrained model/checkpoint that ships an export config. | ||
|
|
||
| **Not implemented yet** — placeholder for a first-class "export recipe" workflow. | ||
|
|
||
| The idea: model owners publish an ``export_config.json`` (or an ``export_config`` field in | ||
| ``config.json``) alongside their weights on the Hub. That file captures the settings the | ||
| owner has already validated for their architecture — the target format (``dynamo`` / | ||
| ``onnx`` / ``executorch``), exact dynamic-shape specs (e.g. ``text_ids`` dynamic to 4096, | ||
| image tiles fixed at 448, ``batch=1`` for edge deployment), ``strict`` flag, ONNX opset, | ||
| prefill vs. decode layout, ExecuTorch backend choice, and any other knob that today lives | ||
| as tribal knowledge in a README or a private notebook. | ||
|
|
||
| Consumers then get the owner-validated export in one call:: | ||
|
|
||
| exporter = AutoHfExporter.from_pretrained("org/model-name") | ||
| program = exporter.export(model, inputs) | ||
|
|
||
| Composes with the [`register_export_input_preparer`] registry: the owner supplies the | ||
| shape spec via ``export_config.json``, transformers supplies the data-dependent | ||
| precomputations (``cu_seqlens``, vision position ids, window indices, …) for that | ||
| architecture. Together they cover the two hard parts of exporting new models — knowing | ||
| the right shape contract and preparing the right inputs — so downstream users don't | ||
| re-derive either from scratch (and don't break in production when they get it wrong). | ||
| """ | ||
| raise NotImplementedError( | ||
| "AutoHfExporter.from_pretrained is not implemented yet. " | ||
| "Load/export configs explicitly and call AutoHfExporter.from_config(...) instead." | ||
| ) | ||
|
|
||
| @staticmethod | ||
| def supports_export_format(export_config_dict: dict) -> bool: | ||
| """Return True if the provided dict describes an ``export_format`` that has both a | ||
| registered config class and a registered exporter class. Warns with an actionable message | ||
| when the format is missing entirely, unknown, or only half-registered.""" | ||
| export_fmt = export_config_dict.get("export_format") | ||
| if export_fmt is None: | ||
| logger.warning( | ||
| "No 'export_format' key in export config — supported values are: " | ||
| f"{sorted(AUTO_EXPORTER_MAPPING)}. Skipping." | ||
| ) | ||
| return False | ||
|
|
||
| name = export_fmt.value if isinstance(export_fmt, ExportFormat) else export_fmt | ||
| has_config = name in AUTO_EXPORT_CONFIG_MAPPING | ||
| has_exporter = name in AUTO_EXPORTER_MAPPING | ||
|
|
||
| if not has_config and not has_exporter: | ||
| logger.warning( | ||
| f"Unknown export format {export_fmt!r} — supported values are: " | ||
| f"{sorted(set(AUTO_EXPORTER_MAPPING) & set(AUTO_EXPORT_CONFIG_MAPPING))}. Skipping." | ||
| ) | ||
| return False | ||
| if not has_config: | ||
| logger.warning( | ||
| f"Export format {name!r} has a registered exporter but no config class. " | ||
| f"Register one via ``@register_export_config({name!r})``. Skipping." | ||
| ) | ||
| return False | ||
| if not has_exporter: | ||
| logger.warning( | ||
| f"Export format {name!r} has a registered config class but no exporter. " | ||
| f"Register one via ``@register_exporter({name!r})``. Skipping." | ||
| ) | ||
| return False | ||
| return True | ||
|
|
||
|
|
||
| def register_exporter(name: str): | ||
| def register_exporter_fn(cls): | ||
| if name in AUTO_EXPORTER_MAPPING: | ||
| logger.warning(f"Exporter '{name}' is already registered and will be overwritten.") | ||
| if not issubclass(cls, HfExporter): | ||
| raise TypeError("Exporter must extend HfExporter") | ||
| AUTO_EXPORTER_MAPPING[name] = cls | ||
| return cls | ||
|
|
||
| return register_exporter_fn | ||
|
|
||
|
|
||
| def register_export_config(name: str): | ||
| def register_export_config_fn(cls): | ||
| if name in AUTO_EXPORT_CONFIG_MAPPING: | ||
| logger.warning(f"Export config '{name}' is already registered and will be overwritten.") | ||
| if not issubclass(cls, ExportConfigMixin): | ||
| raise TypeError("Export config must extend ExportConfigMixin") | ||
| AUTO_EXPORT_CONFIG_MAPPING[name] = cls | ||
| return cls | ||
|
|
||
| return register_export_config_fn | ||
|
|
||
|
|
||
| def get_hf_exporter(export_config) -> HfExporter: | ||
| return AutoHfExporter.from_config(export_config) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.