Scalable Model Merging with Progressive Layerwise Distillation

This repository contains the code of the paper Scalable Model Merging with Progressive Layerwise Distillation.

• The code for Vision tasks can be found in Vision.
• The code for natural language understanding (NLU) tasks can be found in NLU.
• The code for natural language generation (NLG) tasks can be found in NLG.

Acknowledgements: this repository uses codes and resources from Task Arithmetic, TIES Merging, AdaMerging, DARE.