Hallucination detection in foundation models for decision-making: A flexible definition and review of the state of the art

N Chakraborty, M Ornik, K Driggs-Campbell - ACM Computing Surveys, 2024 - dl.acm.org
Autonomous systems are soon to be ubiquitous, spanning manufacturing, agriculture,
healthcare, entertainment, and other industries. Most of these systems are developed with …

Confident adaptive language modeling

T Schuster, A Fisch, J Gupta… - Advances in …, 2022 - proceedings.neurips.cc
Recent advances in Transformer-based large language models (LLMs) have led to
significant performance improvements across many tasks. These gains come with a drastic …

A gentle introduction to conformal prediction and distribution-free uncertainty quantification

AN Angelopoulos, S Bates - arxiv preprint arxiv:2107.07511, 2021 - arxiv.org
Black-box machine learning models are now routinely used in high-risk settings, like
medical diagnostics, which demand uncertainty quantification to avoid consequential model …

Image-to-image regression with distribution-free uncertainty quantification and applications in imaging

AN Angelopoulos, AP Kohli, S Bates… - International …, 2022 - proceedings.mlr.press
Image-to-image regression is an important learning task, used frequently in biological
imaging. Current algorithms, however, do not generally offer statistical guarantees that …

Conformal risk control

AN Angelopoulos, S Bates, A Fisch, L Lei… - arxiv preprint arxiv …, 2022 - arxiv.org
We extend conformal prediction to control the expected value of any monotone loss function.
The algorithm generalizes split conformal prediction together with its coverage guarantee …

Improved online conformal prediction via strongly adaptive online learning

A Bhatnagar, H Wang, C **ong… - … Conference on Machine …, 2023 - proceedings.mlr.press
We study the problem of uncertainty quantification via prediction sets, in an online setting
where the data distribution may vary arbitrarily over time. Recent work develops online …

Conformal language modeling

V Quach, A Fisch, T Schuster, A Yala, JH Sohn… - arxiv preprint arxiv …, 2023 - arxiv.org
We propose a novel approach to conformal prediction for generative language models
(LMs). Standard conformal prediction produces prediction sets--in place of single predictions …

Conformal prediction: A gentle introduction

AN Angelopoulos, S Bates - Foundations and Trends® in …, 2023 - nowpublishers.com
Black-box machine learning models are now routinely used in high-risk settings, like
medical diagnostics, which demand uncertainty quantification to avoid consequential model …

Reliable visual question answering: Abstain rather than answer incorrectly

S Whitehead, S Petryk, V Shakib, J Gonzalez… - … on Computer Vision, 2022 - Springer
Abstract Machine learning has advanced dramatically, narrowing the accuracy gap to
humans in multimodal tasks like visual question answering (VQA). However, while humans …

Robust calibration with multi-domain temperature scaling

Y Yu, S Bates, Y Ma, M Jordan - Advances in Neural …, 2022 - proceedings.neurips.cc
Uncertainty quantification is essential for the reliable deployment of machine learning
models to high-stakes application domains. Uncertainty quantification is all the more …