Adaptive uncertainty estimation via high-dimensional testing on latent representations

TH Chan, KW Lau, J Shen, G Yin… - Advances in Neural …, 2024‏ - proceedings.neurips.cc
Uncertainty estimation aims to evaluate the confidence of a trained deep neural network.
However, existing uncertainty estimation approaches rely on low-dimensional distributional …

A unified framework for dataset shift diagnostics

FM Polo, R Izbicki, EG Lacerda Jr, JP Ibieta-Jimenez… - Information …, 2023‏ - Elsevier
Supervised learning techniques typically assume training data originates from the target
population. Yet, in reality, dataset shift frequently arises, which, if not adequately taken into …

Efficiently mitigating the impact of data drift on machine learning pipelines

S Dong, Q Wang, S Sahri, T Palpanas… - Proceedings of the …, 2024‏ - dl.acm.org
Despite the increasing success of Machine Learning (ML) techniques in real-world
applications, their maintenance over time remains challenging. In particular, the prediction …

" why did the model fail?": Attributing model performance changes to distribution shifts

H Zhang, H Singh, M Ghassemi, S Joshi - 2023‏ - proceedings.mlr.press
Abstract Machine learning models frequently experience performance drops under
distribution shifts. The underlying cause of such shifts may be multiple simultaneous factors …

Empirical data drift detection experiments on real-world medical imaging data

A Kore, E Abbasi Bavil, V Subasri, M Abdalla… - Nature …, 2024‏ - nature.com
While it is common to monitor deployed clinical artificial intelligence (AI) models for
performance degradation, it is less common for the input data to be monitored for data drift …

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives

AA Verma, P Trbovich, M Mamdani… - BMJ Quality & …, 2024‏ - qualitysafety.bmj.com
Machine learning (ML) solutions are increasingly entering healthcare. They are complex,
sociotechnical systems that include data inputs, ML models, technical infrastructure and …

Out of the ordinary: Spectrally adapting regression for covariate shift

B Eyre, E Creager, D Madras, V Papyan… - arxiv preprint arxiv …, 2023‏ - arxiv.org
Designing deep neural network classifiers that perform robustly on distributions differing
from the available training data is an active area of machine learning research. However, out …

Feature attribution explanation to detect harmful dataset shift

Z Wang, C Huang, X Yao - 2023 International Joint Conference …, 2023‏ - ieeexplore.ieee.org
Detecting whether a distribution shift has occurred in the dataset is a critical aspect when
implementing machine learning models, as even a small shift in the data distribution may …

(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancy

E Rosenfeld, S Garg - Advances in Neural Information …, 2024‏ - proceedings.neurips.cc
We derive a new,(almost) guaranteed upper bound on the error of deep neural networks
under distribution shift using unlabeled test data. Prior methods are either vacuous in …

A Geometric Explanation of the Likelihood OOD Detection Paradox

H Kamkari, BL Ross, JC Cresswell, AL Caterini… - arxiv preprint arxiv …, 2024‏ - arxiv.org
Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour:
when trained on a relatively complex dataset, they assign higher likelihood values to out-of …