Upgrade to transformers release V4.23.1 #62

KSGulin · 2022-10-14T00:26:10Z

Note: currently targeted at copy of upstream release branch for easy comparison. Diff is intended to land on our main branch.

This PR integrates our existing transformers integration with the updated v4.23.1 upstream. See here for HuggingFace release notes.

Meant to land concurrently with neuralmagic/sparseml#1081

Testing Plan

Integration tests (both cadences)
SparseML QA docs training and export commands
One training command from the text classification and token classification docs each

HuggingFaceDocBuilderDev · 2022-10-14T00:39:22Z

The documentation is not available anymore as the PR was closed or merged.

corey-nm · 2022-10-14T14:02:58Z

src/transformers/models/bert/modeling_bert.py

Yeah I'm almost positive we could hot patch this stuff in instead of having repo forks.

Something like

import transformers from transformers.models.bert.modeling_bert import BertSelfAttention as _BertSelfAttention class PatchedBertSelfAttention(_BertSelfAttention): def __init__(self, *args, **kwargs): self.attention_scores_matmul = ... def forward(self): ... transformers.models.bert.modeling_bert.BertSelfAttention = PatchedBertSelfAttention

Yeah I think now that most of the implementation has been moved on the sparseml side, there's definitely potential to explore here. One thing to keep in mind is that every time we've upgraded the HF repo had changes that broke our integration, so the hot patch would need to be easy to debug and amend. But in general I'm all for this

corey-nm · 2022-10-14T14:04:16Z

src/transformers/sparse.py

What was the reasoning for adding this class? Was this functionality not here before?

This is a file I noted for myself to remove before pushing and did not follow through

Disable FP16 on QAT start (#12) * Override LRScheduler when using LRModifiers * Disable FP16 on QAT start * keep wrapped scaler object for training after disabling Using QATMatMul in DistilBERT model class (#41) Removed double quantization of output of context layer. (#45) Fix DataParallel validation forward signatures (#47) * Fix: DataParallel validation forward signatures * Update: generalize forward_fn selection Best model after epoch (#46) fix sclaer check for non fp16 mode in trainer (#38) Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. enable a QATWrapper for non-parameterized matmuls in BERT self attention (#9)

update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names

KSGulin · 2022-10-14T18:22:20Z

Committed the sin of editing history and paired down our fork changes to 3 commits

corey-nm · 2022-10-14T18:26:33Z

src/transformers/hf_argparser.py

+                f"files downloaded from {val}. Found {framework_file_names}. Check "
+                "if the given stub is for a transformers repo model"
+            )
+        framework_dir_path = Path(framework_file_paths[0]).parent.absolute()


Will all the framework_file_paths have the same parent? Is that why we can just use the 1st one?

corey-nm · 2022-10-14T18:27:30Z

src/transformers/hf_argparser.py

+        return tuple(
+            [_download_dataclass_zoo_stub_files(output) for output in outputs]
+        )


not sure if i like this better but thought i'd add

Suggested change

return tuple(

[_download_dataclass_zoo_stub_files(output) for output in outputs]

)

return tuple(map(_download_dataclass_zoo_stub_files, outputs))

corey-nm · 2022-10-14T18:29:16Z

src/transformers/models/distilbert/modeling_distilbert.py

+        self.wrap_qat = True
+        self.qat_wrapper_kwargs = {
+            "num_inputs": 2,
+            "num_outputs": 0,


Is this supposed to be here? Don't see it in others

bfineran · 2022-10-18T14:22:36Z

@corey-nm good comments, going to keep changes to existing flows out of scope for this PR as it's mean to be a rebase only, let's dig in together offline

* Add recipe_name to default file names * Upgrade to transformers release V4.30.2 (#62) * Update trainer and model flows to accommodate sparseml Disable FP16 on QAT start (#12) * Override LRScheduler when using LRModifiers * Disable FP16 on QAT start * keep wrapped scaler object for training after disabling Using QATMatMul in DistilBERT model class (#41) Removed double quantization of output of context layer. (#45) Fix DataParallel validation forward signatures (#47) * Fix: DataParallel validation forward signatures * Update: generalize forward_fn selection Best model after epoch (#46) fix sclaer check for non fp16 mode in trainer (#38) Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. enable a QATWrapper for non-parameterized matmuls in BERT self attention (#9) * Utils and auxillary changes update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names * Fix errors introduced in manual cherry-pick upgrade Co-authored-by: Benjamin Fineran <[email protected]> * update build versions for NM fork pypi push (#74) * fix nightly package name (#75) * add make build command (#76) * add GHA workflow files to build nightly and release packages (#77) * add GHA workflow files to build nightly and release packages * fix name --------- Co-authored-by: dhuang <[email protected]> * bump up version to 1.6.0 (#79) Co-authored-by: dhuang <[email protected]> --------- Co-authored-by: Konstantin <[email protected]> Co-authored-by: Konstantin Gulin <[email protected]> Co-authored-by: dhuangnm <[email protected]> Co-authored-by: dhuang <[email protected]>

(previous commits) * Add recipe_name to default file names * Upgrade to transformers release V4.30.2 (#62) * Update trainer and model flows to accommodate sparseml Disable FP16 on QAT start (#12) * Override LRScheduler when using LRModifiers * Disable FP16 on QAT start * keep wrapped scaler object for training after disabling Using QATMatMul in DistilBERT model class (#41) Removed double quantization of output of context layer. (#45) Fix DataParallel validation forward signatures (#47) * Fix: DataParallel validation forward signatures * Update: generalize forward_fn selection Best model after epoch (#46) fix sclaer check for non fp16 mode in trainer (#38) Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. enable a QATWrapper for non-parameterized matmuls in BERT self attention (#9) * Utils and auxillary changes update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names * Fix errors introduced in manual cherry-pick upgrade Co-authored-by: Benjamin Fineran <[email protected]> * update build versions for NM fork pypi push (#74) * fix nightly package name (#75) * add make build command (#76) * add GHA workflow files to build nightly and release packages (#77) * add GHA workflow files to build nightly and release packages * fix name --------- Co-authored-by: dhuang <[email protected]> * bump up version to 1.6.0 (#79) Co-authored-by: dhuang <[email protected]> --------- Co-authored-by: Konstantin <[email protected]> Co-authored-by: Konstantin Gulin <[email protected]> Co-authored-by: dhuangnm <[email protected]> Co-authored-by: dhuang <[email protected]> minor improvements for build workflow files (#83) Co-authored-by: dhuang <[email protected]> fix minor issue (#84) Co-authored-by: dhuang <[email protected]> OPT with quantizable MatMuls (#85) fix a minor issue for release build (#86) Co-authored-by: dhuang <[email protected]> update version in version.py Testmo (#91) * improve GHA workflow files to build nightly and release, and report status to testmo * clean up * report exit code * Assign value to exit_code --------- Co-authored-by: dhuang <[email protected]> Update trainer.py - fix DistributedSampler import (#93) DistributedSampler is used but not imported in `trainer.py` Research/llama/bmm quantization (#94) * Quantize attention matmuls * Quantize attention matmuls bump base transformers version

KSGulin self-assigned this Oct 14, 2022

KSGulin mentioned this pull request Oct 14, 2022

Updates needed to support transformers v4.23.1 neuralmagic/sparseml#1081

Merged

KSGulin requested review from a team, Chibukach, DaltheCow and corey-nm and removed request for a team October 14, 2022 10:43

KSGulin changed the title ~~Upgrade to transformers release V4.23~~ Upgrade to transformers release V4.23.1 Oct 14, 2022

KSGulin added the mle-team label Oct 14, 2022

corey-nm reviewed Oct 14, 2022

View reviewed changes

KSGulin force-pushed the upstream_sync_4.23 branch from 9282e09 to 6133554 Compare October 14, 2022 17:38

bfineran and others added 3 commits October 14, 2022 18:17

Utils and auxillary changes

05297a8

update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names

Fix errors introduced in manual cherry-pick upgrade

7548869

KSGulin force-pushed the upstream_sync_4.23 branch from 6133554 to 7548869 Compare October 14, 2022 18:18

bfineran approved these changes Oct 14, 2022

View reviewed changes

bfineran requested a review from anmarques October 14, 2022 18:25

corey-nm reviewed Oct 14, 2022

View reviewed changes

bfineran merged commit 46f78c1 into upstream-v4.23-release-copy Oct 18, 2022

DaltheCow removed their request for review October 18, 2022 14:36

dbogunowicz deleted the upstream_sync_4.23 branch December 5, 2023 10:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Upgrade to transformers release V4.23.1 #62

Upgrade to transformers release V4.23.1 #62

Uh oh!

KSGulin commented Oct 14, 2022 •

edited

Loading

Uh oh!

HuggingFaceDocBuilderDev commented Oct 14, 2022 •

edited

Loading

Uh oh!

corey-nm Oct 14, 2022 •

edited

Loading

Uh oh!

KSGulin Oct 14, 2022 •

edited

Loading

Uh oh!

corey-nm Oct 14, 2022

Uh oh!

KSGulin Oct 14, 2022

Uh oh!

KSGulin commented Oct 14, 2022

Uh oh!

corey-nm Oct 14, 2022

Uh oh!

corey-nm Oct 14, 2022

Uh oh!

corey-nm Oct 14, 2022

Uh oh!

bfineran commented Oct 18, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Upgrade to transformers release V4.23.1 #62

Upgrade to transformers release V4.23.1 #62

Uh oh!

Conversation

KSGulin commented Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corey-nm Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KSGulin Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corey-nm Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

KSGulin Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

KSGulin commented Oct 14, 2022

Uh oh!

corey-nm Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

corey-nm Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

corey-nm Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

bfineran commented Oct 18, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

KSGulin commented Oct 14, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Oct 14, 2022 •

edited

Loading

corey-nm Oct 14, 2022 •

edited

Loading

KSGulin Oct 14, 2022 •

edited

Loading