AI alignment aims to make AI systems behave in line with human intentions and values. As
AI systems grow more capable, so do risks from misalignment. To provide a comprehensive …

In recent years, the development of robotics and artificial intelligence (AI) systems has been
nothing short of remarkable. As these systems continue to evolve, they are being utilized in …

Reinforcement learning from human feedback (RLHF) is a variant of reinforcement learning
(RL) that learns from human feedback instead of relying on an engineered reward function …

Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a
crucial challenge, especially for AI systems with a high degree of autonomy and general …

Inverse reinforcement learning (IRL) aims to recover the reward function and the associated
optimal policy that best fits observed sequences of states and actions implemented by an …

It is often very challenging to manually design reward functions for complex, real-world
tasks. To solve this, one can instead use reward learning to infer a reward function from …

Abstract The aim of Inverse Reinforcement Learning (IRL) is to infer a reward function R from
a policy pi. To do this, we need a model of how pi relates to R. In the current literature, the …

The dominant practice of AI alignment assumes (1) that preferences are an adequate
representation of human values,(2) that human rationality can be understood in terms of …

Inverse reinforcement learning attempts to reconstruct the reward function in a Markov
decision problem, using observations of agent actions. As already observed in Russell …

The utility of reinforcement learning is limited by the alignment of reward functions with the
interests of human stakeholders. One promising method for alignment is to learn the reward …