Reinforcement learning from human feedback (RLHF) is a technique for training AI systems
to align with human goals. RLHF has emerged as the central method used to finetune state …

Reinforcement learning from human feedback (RLHF) is a variant of reinforcement learning
(RL) that learns from human feedback instead of relying on an engineered reward function …

Learning policies via preference-based reward learning is an increasingly popular method
for customizing agent behavior, but has been shown anecdotally to be prone to spurious …

Although the notion of task similarity is potentially interesting in a wide range of areas such
as curriculum learning or automated planning, it has mostly been tied to transfer learning …

In order to solve a task using reinforcement learning, it is necessary to first formalise the goal
of that task as a reward function. However, for many real-world tasks, it is very difficult to …

The success of Reinforcement Learning from Human Feedback (RLHF) in language model
alignment is critically dependent on the capability of the reward model (RM). However, as …

Inverse reinforcement learning (IRL) aims to infer an agent's preferences (represented as a
reward function $ R $) from their behaviour (represented as a policy $\pi $). To do this, we …

Preference-based reward learning is a popular technique for teaching robots and
autonomous systems how a human user wants them to perform a task. Previous works have …

In reward learning, it is helpful to be able to measure distances between reward functions,
for example to evaluate learned reward models. Using simple metrics such as L^ 2 …

The aim of Inverse Reinforcement Learning (IRL) is to infer a reward function $ R $ from a
policy $\pi $. This problem is difficult, for several reasons. First of all, there are typically …