Uncertainty quantification (UQ) methods play a pivotal role in reducing the impact of
uncertainties during both optimization and decision making processes. They have been …

Validation metrics are key for tracking scientific progress and bridging the current chasm
between artificial intelligence research and its translation into practice. However, increasing …

Language models (LMs) are becoming the foundation for almost all major language
technologies, but their capabilities, limitations, and risks are not well understood. We present …

Do large language models (LLMs) know the law? LLMs are increasingly being used to
augment legal practice, education, and research, yet their revolutionary potential is …

Machine learning (ML) systems are rapidly increasing in size, are acquiring new
capabilities, and are increasingly deployed in high-stakes settings. As with other powerful …

Accurate estimation of predictive uncertainty (model calibration) is essential for the safe
application of neural networks. Many instances of miscalibration in modern neural networks …

Estimating the difficulty of a dataset typically involves comparing state-of-the-art models to
humans; the bigger the performance gap, the harder the dataset is said to be. However, this …

Bayesian neural networks (BNN) and deep ensembles are principled approaches to
estimate the predictive uncertainty of a deep learning model. However their practicality in …

Miscalibration--a mismatch between a model's confidence and its correctness--of Deep
Neural Networks (DNNs) makes their predictions hard to rely on. Ideally, we want networks …

We introduce two challenging datasets that reliably cause machine learning model
performance to substantially degrade. The datasets are collected with a simple adversarial …