Reinforcement Learning from Human Feedback (RLHF) is a powerful paradigm for aligning
foundation models to human values and preferences. However, current RLHF techniques …

Alignment of the language model with human preferences is a common approach to making
a language model useful to end users. However, most alignment work is done in English …

Frontier AI systems, including large language models (LLMs), hold increasing influence over
the epistemology of human users. Such influence can reinforce prevailing societal values …

Reinforcement learning from human feedback (RLHF) has become an essential step in fine-
tuning large language models (LLMs) to align them with human preferences. However …

LLMs are increasingly used to design reward functions based on human preferences in
Reinforcement Learning (RL). We focus on LLM-designed rewards for Restless Multi-Armed …

Social choice theory is the study of preference aggregation across a population, used both
in mechanism design for human agents and in the democratic alignment of language …

LLMs are tuned to follow instructions (aligned) by learning which of two outputs users prefer
for a prompt. However, this preference data format does not convey why users prefer …

RLHF has emerged as a pivotal step in aligning language models with human objectives
and values. It typically involves learning a reward model from human preference data and …

Ensuring that large language models (LLMs) reflect diverse user values and preferences is
crucial as their user bases expand globally. It is therefore encouraging to see the growing …

The deployment of AI systems for welfare benefit allocation allows for accelerated decision-
making and faster provision of critical help, but has already led to an increase in unfair …