System Info
transformers v4.43.1
Who can help?
@ArthurZucker @gante
Information
Tasks
Reproduction
Using an older rope_config with new transformers version
Expected behavior
Hey, there seems to be an issue when trying to use an older model that has the previous rope_config. Specifically, the rope_config used to have the "type" key, and while this has now been changed to "rope_type", the key is not being extracted correctly. At this line, the "type" key is extracted for backwards compatibility but is not set inside the rope_scaling dict. And all the validation functions expect rope_scaling to have "rope_type" as a key (for example, here).
Could this be addressed? Would likely require another patch release as well. Thank you!
System Info
transformers v4.43.1
Who can help?
@ArthurZucker @gante
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
Using an older rope_config with new transformers version
Expected behavior
Hey, there seems to be an issue when trying to use an older model that has the previous rope_config. Specifically, the rope_config used to have the "type" key, and while this has now been changed to "rope_type", the key is not being extracted correctly. At this line, the "type" key is extracted for backwards compatibility but is not set inside the
rope_scalingdict. And all the validation functions expectrope_scalingto have "rope_type" as a key (for example, here).Could this be addressed? Would likely require another patch release as well. Thank you!