Skip to content

Add Mamba2ForSequenceClassficiation#1

Merged
Adibvafa merged 19 commits into
mainfrom
adib-dev
Jul 17, 2024
Merged

Add Mamba2ForSequenceClassficiation#1
Adibvafa merged 19 commits into
mainfrom
adib-dev

Conversation

@Adibvafa

Copy link
Copy Markdown

No description provided.

@Adibvafa
Adibvafa requested a review from vasqu July 13, 2024 21:29
@Adibvafa

Copy link
Copy Markdown
Author

@vasqu I did also change the cache parameter name to cache_params for consistency with Mamba implementation. Please take a look, if ok merge.

@vasqu vasqu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks overall good. Fixing the init should be a priority so that the sequence classification model can be imported from transformers.

Comment thread src/transformers/models/mamba2/configuration_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/modeling_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/modeling_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/modeling_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/modeling_mamba2.py
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
Comment thread tests/models/mamba2/test_modeling_mamba2.py
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/__init__.py Outdated
@vasqu

vasqu commented Jul 13, 2024

Copy link
Copy Markdown
Collaborator

Last thing but can we also add the new sequence model to dummy_pt_objects?

Comment thread tests/models/mamba2/test_modeling_mamba2.py
@Adibvafa
Adibvafa requested a review from vasqu July 15, 2024 15:37
@Adibvafa

Copy link
Copy Markdown
Author

@vasqu Thank you so much for the detailed PR review! I resolved most comments, could you take a look?

@vasqu

vasqu commented Jul 15, 2024

Copy link
Copy Markdown
Collaborator

@Adibvafa Will take a closer relook in a bit and good work!

@vasqu vasqu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the stuff I commented on otherwise LGTM!

@Adibvafa
Adibvafa requested a review from vasqu July 15, 2024 23:37

@vasqu vasqu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests seem to fail and I'm not sure why. Could you look into it? Not all tests should pass, but especially the ones already written (i.e. the ones that also pass in main).

Let's revert some stuff and really focus on only adding the sequence classifier first and handle renames etc separately.

Comment thread tests/models/mamba/test_modeling_mamba.py
Comment thread src/transformers/models/mamba2/modeling_mamba2.py Outdated
Comment thread src/transformers/models/mamba2/modeling_mamba2.py
Comment thread tests/models/mamba/test_modeling_mamba.py Outdated
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
Comment thread tests/models/mamba2/test_modeling_mamba2.py Outdated
@Adibvafa
Adibvafa merged commit c535499 into main Jul 17, 2024
@Adibvafa
Adibvafa deleted the adib-dev branch July 17, 2024 17:43
vasqu added a commit that referenced this pull request Jul 18, 2024
Some fixes for the stuff introduced in PR #1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants