Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

Model merging aims to combine multiple expert models into a more capable single model,
offering benefits such as reduced storage and serving costs, improved generalization, and …

In this paper, we study format biases in reinforcement learning from human feedback
(RLHF). We observe that many widely-used preference models, including human …

Model merging combines multiple expert models-finetuned from a base foundation model
on diverse tasks and domains-into a single, more capable model. However, most existing …

Model merging combines knowledge from task-specific models into a unified multi-task
model to avoid joint training on all task data. However, current methods face challenges due …

Model merging has shown great promise at combining expert models, but the benefit of
merging is unclear when merging``generalist''models trained on many tasks. We explore …

The rapid proliferation of domain-specialized machine learning models presents a
challenge: while individual models excel in specific domains, their performance varies …

Model merging has emerged as an effective approach to combine multiple single-task
models into a multitask model. This process typically involves computing a weighted …