Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

Fusion is a technique for merging multiple independently-trained neural networks in order to
combine their capabilities. Past attempts have been restricted to the case of fully-connected …

Neural networks can be significantly compressed by pruning, leading to sparse models
requiring considerably less storage and floating-point operations while maintaining …

The scalability of deep learning models is fundamentally limited by computing resources,
memory, and communication. Although methods like low-rank adaptation (LoRA) have …

Model merging offers an effective strategy to combine the strengths of multiple finetuned
models into a unified model that preserves the specialized capabilities of each. Existing …

We focus on the problem of fusing two or more heterogeneous large language models
(LLMs) to facilitate their complementary strengths. One of the challenges on model fusion is …

Model merging has recently emerged as a cost-efficient paradigm for multi-task learning.
Among current approaches, task arithmetic stands out for its simplicity and effectiveness. In …

The multi-modal nature of neural loss landscapes is often considered to be the main driver
behind the empirical success of deep ensembles. In this work, we probe this belief by …

The fine-tuning of deep pre-trained models has recently revealed compositional properties.
This enables the arbitrary composition of multiple specialized modules into a single, multi …