Empirical data drift detection experiments on real-world medical imaging data

A Kore, E Abbasi Bavil, V Subasri, M Abdalla… - Nature …, 2024 - nature.com
While it is common to monitor deployed clinical artificial intelligence (AI) models for
performance degradation, it is less common for the input data to be monitored for data drift …

A geometric explanation of the likelihood OOD detection paradox

H Kamkari, BL Ross, JC Cresswell, AL Caterini… - arxiv preprint arxiv …, 2024 - arxiv.org
Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour:
when trained on a relatively complex dataset, they assign higher likelihood values to out-of …

A unified framework for dataset shift diagnostics

FM Polo, R Izbicki, EG Lacerda Jr, JP Ibieta-Jimenez… - Information …, 2023 - Elsevier
Supervised learning techniques typically assume training data originates from the target
population. Yet, in reality, dataset shift frequently arises, which, if not adequately taken into …

Efficiently mitigating the impact of data drift on machine learning pipelines

S Dong, Q Wang, S Sahri, T Palpanas… - Proceedings of the …, 2024 - dl.acm.org
Despite the increasing success of Machine Learning (ML) techniques in real-world
applications, their maintenance over time remains challenging. In particular, the prediction …

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives

AA Verma, P Trbovich, M Mamdani… - BMJ Quality & …, 2024 - qualitysafety.bmj.com
Machine learning (ML) solutions are increasingly entering healthcare. They are complex,
sociotechnical systems that include data inputs, ML models, technical infrastructure and …

Adaptive uncertainty estimation via high-dimensional testing on latent representations

TH Chan, KW Lau, J Shen, G Yin… - Advances in Neural …, 2023 - proceedings.neurips.cc
Uncertainty estimation aims to evaluate the confidence of a trained deep neural network.
However, existing uncertainty estimation approaches rely on low-dimensional distributional …

Feature attribution explanation to detect harmful dataset shift

Z Wang, C Huang, X Yao - 2023 International Joint Conference …, 2023 - ieeexplore.ieee.org
Detecting whether a distribution shift has occurred in the dataset is a critical aspect when
implementing machine learning models, as even a small shift in the data distribution may …

(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancy

E Rosenfeld, S Garg - Advances in Neural Information …, 2023 - proceedings.neurips.cc
We derive a new,(almost) guaranteed upper bound on the error of deep neural networks
under distribution shift using unlabeled test data. Prior methods are either vacuous in …

Trimming the Risk: Towards Reliable Continuous Training for Deep Learning Inspection Systems

AA Abbassi, HB Braiek, F Khomh, T Reid - arxiv preprint arxiv:2409.09108, 2024 - arxiv.org
The industry increasingly relies on deep learning (DL) technology for manufacturing
inspections, which are challenging to automate with rule-based machine vision algorithms …