This work identifies 18 foundational challenges in assuring the alignment and safety of large
language models (LLMs). These challenges are organized into three different categories …

Abstract Large Language Models (LLMs) have recently gained significant attention due to
their remarkable capabilities in performing diverse tasks across various domains. However …

Protein language models (pLMs) trained on large protein sequence databases have been
used to understand disease and design novel proteins. In design tasks, the likelihood of a …

Companies, organizations, and governments increasingly exploit Language Models'(LM)
remarkable capability to display agent-like behavior. As LMs are adopted to perform tasks …

Reinforcement Learning from Human Feedback (RLHF) has proven effective in aligning
large language models with human intentions, yet it often relies on complex methodologies …

We examine multi-task benchmarks in machine learning through the lens of social choice
theory. We draw an analogy between benchmarks and electoral systems, where models are …

This application paper introduces a transformative solution to address the labour-intensive
manual report generation, data searching & report revision process in government entities …

Building a reliable and automated evaluation metric is a necessary but challenging problem
for open-domain dialogue systems. Recent studies proposed evaluation metrics that assess …

Human evaluation of generated language through pairwise preference judgments is
pervasive. However, under common scenarios, such as when generations from a model pair …

Large language models are often ranked according to their level of alignment with human
preferences--a model is better than other models if its outputs are more frequently preferred …