Model merging aims to combine multiple expert models into a more capable single model,
offering benefits such as reduced storage and serving costs, improved generalization, and …

Deep model training on extensive datasets is increasingly becoming cost-prohibitive,
prompting the widespread adoption of deep model fusion techniques to leverage knowledge …

Fine-tuning large language models (LLMs) with low-rank adaptations (LoRAs) has become
common practice, often yielding numerous copies of the same LLM differing only in their …

Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and
facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates …

Understanding neural networks is crucial to creating reliable and trustworthy deep learning
models. Most contemporary research in interpretability analyzes just one model at a time via …

Low-Rank Adaptation (LoRA) has emerged as a popular technique for fine-tuning large
language models (LLMs) to various domains due to its modular design and widespread …

Model merging-based multitask learning (MTL) offers a promising approach for performing
MTL by merging multiple expert models without requiring access to raw training data …

In this work, we explore the limitations of combining models by averaging intermediate
features, referred to as model merging, and propose a new direction for achieving collective …

Model merging combines multiple expert models-finetuned from a base foundation model
on diverse tasks and domains-into a single, more capable model. However, most existing …

Deep model merging represents an emerging research direction that combines multiple fine-
tuned models to harness their specialized capabilities across different tasks and domains …