Model merging is an efficient empowerment technique in the machine learning community
that does not require the collection of raw training data and does not require expensive …

The remarkable success of Large Language Models (LLMs) has ushered natural language
processing (NLP) research into a new era. Despite their diverse capabilities, LLMs trained …

This paper surveys research in the rapidly growing field of enhancing large language
models (LLMs) with reinforcement learning (RL), a technique that enables LLMs to improve …

With the rapid advancement of large language models (LLMs), aligning policy models with
human preferences has become increasingly critical. Direct Preference Optimization (DPO) …

Direct Preference Optimization (DPO) has emerged as a prominent algorithm for the direct
and robust alignment of Large Language Models (LLMs) with human preferences, offering a …

In this paper, we study format biases in reinforcement learning from human feedback
(RLHF). We observe that many widely-used preference models, including human …

Recently, both closed-source LLMs and open-source communities have made significant
strides, outperforming humans in various general domains. However, their performance in …

The task of multi-objective alignment aims at balancing and controlling the different
alignment objectives (eg, helpfulness, harmlessness and honesty) of large language models …

As large language models (LLMs) are overwhelmingly more and more integrated into
various applications, ensuring they generate safe and aligned responses is a pressing …

While learning to align Large Language Models (LLMs) with human preferences has shown
remarkable success, aligning these models to meet the diverse user preferences presents …