Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities

E Yang, L Shen, G Guo, X Wang, X Cao… - arxiv preprint arxiv …, 2024 - arxiv.org
Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

Lines: Post-training layer scaling prevents forgetting and enhances model merging

K Wang, N Dimitriadis, A Favero… - arxiv preprint arxiv …, 2024 - arxiv.org
Large pre-trained models exhibit impressive zero-shot performance across diverse tasks,
but fine-tuning often leads to catastrophic forgetting, where improvements on a target …

RandLoRA: Full-rank parameter-efficient fine-tuning of large models

P Albert, FZ Zhang, H Saratchandran… - arxiv preprint arxiv …, 2025 - arxiv.org
Low-Rank Adaptation (LoRA) and its variants have shown impressive results in reducing the
number of trainable parameters and memory requirements of large transformer networks …

I-Lora: Iterative Merging of Routing-Tuned Low-Rank Adapters for Multi-task Learning

G Zhao, Q Zhang, S Zhai, D Shen, Y Qiao, T Xu - openreview.net
The advancement of vision-language models has significantly boosted the performance of
embodied and game AI, endowing them with more robust general visual understanding …