Aligning large language models (LLMs) with human preferences through reinforcement
learning (RLHF) can lead to reward hacking, where LLMs exploit failures in the reward …

A common explanation for the failure of out-of-distribution (OOD) generalization is that the
model trained with empirical risk minimization (ERM) learns spurious features instead of …

LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under
Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained …

It has been widely observed that deep neural networks (DNN) are vulnerable to backdoor
attacks where attackers could manipulate the model behavior maliciously by tampering with …

Reinforcement learning from human feedback (RLHF) aligns large language models (LLMs)
by encouraging their generations to have high rewards, using a reward model trained on …

Reinforcement Learning from Human Feedback (RLHF) is an effective approach for aligning
language models to human preferences. Central to RLHF is learning a reward function for …

Adapting a pre-trained foundation model on downstream tasks should ensure robustness
against distribution shifts without the need to retrain the whole model. Although existing …

Invariance learning methods aim to learn invariant features in the hope that they generalize
under distributional shifts. Although many tasks are naturally characterized by continuous …

Successful out-of-distribution generalization requires environment annotations.
Unfortunately, these are resource-intensive to obtain, and their relevance to model …

Distribution shifts hinder the deployment of deep learning in real-world problems.
Distribution shifts appear when train and test data come from different sources, which …