Add `MBart` support for `BetterTransformer` by ravenouse · Pull Request #516 · huggingface/optimum

ravenouse · 2022-11-26T21:18:40Z

This PR adds the MBartEncoderLayerBetterTransformer class to support MBart.

During testing, the code is runnable but it yields different results compared with the original transformer model, as shown in the below picture. The specific models I tested are facebook/mbart-large-50 and facebook/mbart-large-cc25. The downstream task I tested is fill masks

Could you tell me what I can do to solve this problem and what other tests I need to run?

Thank you so much!

younesbelkada

Hi @ravenouse !
Thanks a lot for your PR! Glad that the conversion worked! We're almost there!
It seems that MBart uses pre-attention layer norm, could you try to set the attribute to True (as suggested on the suggestion below)
Let me know how it goes!

Co-authored-by: Younes Belkada <[email protected]>

ravenouse · 2022-11-27T00:29:52Z

Hi @ravenouse ! Thanks a lot for your PR! Glad that the conversion worked! We're almost there! It seems that MBart uses pre-attention layer norm, could you try to set the attribute to True (as suggested on the suggestion below) Let me know how it goes!

Hi Younes, thank you so much for the advice! It worked! This time the bt pipeline yields the exactly same results with
the original transformer pipeline!

Please let me know what else needed to be done!

Thanks again!

younesbelkada · 2022-11-27T09:26:00Z

Very glad it worked @ravenouse ! Can't wait to see this PR to be merged!
Let me guide you step by step regarding the immediate next steps:

Step 1: Finish the integration for MBart

Since the refactoring for encoder-decoder based models tests has been recently addressed, adding the tests for MBart should be very easy. Just add a line with "hf-internal-testing/tiny-random-MBartModel" on the list ALL_ENCODER_DECODER_MODELS_TESTS here - push this change to see if the test pass (it should pass if everything works as expected , or you can just run the command pytest tests/bettertransformer/test_bettertransformer_encoder.py::BetterTransformersEncoderDecoderTest)
Add a line stating that MBart is supported for BetterTransformer on the documentation. The addition should go here (make sure to respect the alphabetical order)

Step 2: Add the integration for NLLB

NLLB is a recent encoder-decoder model from Meta AI that is integrated in transformers. Since NLLB uses the same encoder layer as MBart (see this line) the addition is very easy. Just add the line "M2M100EncoderLayer":MBartEncoderLayerBetterTransformer here.
Once the model is integrated as follows. Follow the same procedure as Step 1, and you can use the model hf-internal-testing/tiny-random-nllb for the integration tests ;)
Let me know if anything is unclear! Again thanks a lot for your great contribution @ravenouse 💪

ravenouse · 2022-11-29T02:05:18Z

Hi Younes, thank you so much for your detailed instructions! I have made all changes you mentioned above and I have run the pytest twice locally.

Please let me know what else is needed to be done.

younesbelkada

Thanks so much @ravenouse !
I left a small comment, could you please update ALL_ENCODER_DECODER_MODELS instead so that the tests will run on the new models? Of course let me know how it goes!

younesbelkada · 2022-11-29T09:27:38Z

+    "hf-internal-testing/tiny-random-MBartModel",
+    "hf-internal-testing/tiny-random-nllb",


I think that you have to move them to ALL_ENCODER_DECODER_MODELS: the test pytest tests/bettertransformer/test_bettertransformer_encoder.py::BetterTransformersEncoderDecoderTest will only run on the models listed on ALL_ENCODER_DECODER_MODELS ;)

Hi @younesbelkada. Thank you so much for the explanations! Now I have a better understanding of what's going on in the test files. I have moved the two test models to the right list. I run the pytest again and pass it.

Please let me know what else I can do!

HuggingFaceDocBuilderDev · 2022-11-30T06:33:47Z

The documentation is not available anymore as the PR was closed or merged.

younesbelkada · 2022-11-30T07:13:25Z

Thanks a lot!
Could you just run the styling (run "make style") push the changes and we should be good to merge! 💪🏻

ravenouse · 2022-11-30T07:34:27Z

Hi @younesbelkada. I have run the make style command and reformatted the code. Please let me know what you think about it!

younesbelkada

Looking great!
Thanks a lot for your clean implementation of M2M100 support for BetterTransformer 💪
Looking forward to your next contributions ;)

michaelbenayoun

LGTM!

ravenouse · 2022-12-01T01:39:01Z

Hi @younesbelkada and @michaelbenayoun !
Thank you so much for all the help provided! It is my first PR and it is really an amazing and extremely beneficial experience for me!
I am wondering if I can choose another model to work on. Namely, the ASTLayer. I think the progress will be more smooth..

younesbelkada · 2022-12-01T09:06:12Z

Thanks so much @ravenouse !
Sure yes, just make sure to notify other contributors in huggingface/transformers#20372 too ;)

ravenouse added 2 commits November 26, 2022 00:51

added MBartEncoderLayerBetterTransformer

255251f

fixed some bugs

d68a8f4

younesbelkada reviewed Nov 26, 2022

View reviewed changes

Comment thread optimum/bettertransformer/models/encoder_models.py Outdated

This was referenced Nov 26, 2022

Community contribution - BetterTransformer integration for more models! huggingface/transformers#20372

Closed

Community contribution - BetterTransformer integration for more models! #488

Open

Update optimum/bettertransformer/models/encoder_models.py

478473f

Co-authored-by: Younes Belkada <[email protected]>

younesbelkada mentioned this pull request Nov 27, 2022

Add BetterTransformers support for M2M100 models #521

Closed

ravenouse added 5 commits November 28, 2022 18:24

added test and doc for the mbart implementation

89274c6

ended merge

4546193

fixed the mistake made in the last push

b93a3ea

fixed the mistakes in the previous steps

58377cc

added test and doc for M2M100

77b3467

younesbelkada reviewed Nov 29, 2022

View reviewed changes

moved the test cases/models for the right list

3c1b345

reformatted the code

d0720cc

younesbelkada approved these changes Nov 30, 2022

View reviewed changes

younesbelkada requested a review from michaelbenayoun November 30, 2022 08:34

michaelbenayoun approved these changes Nov 30, 2022

View reviewed changes

younesbelkada merged commit 31eb67c into huggingface:main Nov 30, 2022

		"hf-internal-testing/tiny-random-MBartModel",
		"hf-internal-testing/tiny-random-nllb",

Uh oh!

Conversation

ravenouse commented Nov 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

younesbelkada left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ravenouse commented Nov 27, 2022

Uh oh!

younesbelkada commented Nov 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ravenouse commented Nov 29, 2022

Uh oh!

younesbelkada left a comment

Choose a reason for hiding this comment

Uh oh!

younesbelkada Nov 29, 2022

Choose a reason for hiding this comment

Uh oh!

ravenouse Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Nov 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

younesbelkada commented Nov 30, 2022

Uh oh!

ravenouse commented Nov 30, 2022

Uh oh!

younesbelkada left a comment

Choose a reason for hiding this comment

Uh oh!

michaelbenayoun left a comment

Choose a reason for hiding this comment

Uh oh!

ravenouse commented Dec 1, 2022

Uh oh!

younesbelkada commented Dec 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ravenouse commented Nov 26, 2022 •

edited

Loading

younesbelkada commented Nov 27, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Nov 30, 2022 •

edited

Loading