The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

D Schwabe, K Becker, M Seyferth, A Klaß… - NPJ Digital …, 2024 - nature.com
The adoption of machine learning (ML) and, more specifically, deep learning (DL)
applications into all major areas of our lives is underway. The development of trustworthy AI …

Dataperf: Benchmarks for data-centric ai development

M Mazumder, C Banbury, X Yao… - Advances in …, 2023 - proceedings.neurips.cc
Abstract Machine learning research has long focused on models rather than datasets, and
prominent datasets are used for common ML tasks without regard to the breadth, difficulty …

Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks

B Mucsányi, M Kirchhof, SJ Oh - Advances in Neural …, 2025 - proceedings.neurips.cc
Uncertainty quantification, once a singular task, has evolved into a spectrum of tasks,
including abstained prediction, out-of-distribution detection, and aleatoric uncertainty …

Eliciting and learning with soft labels from every annotator

KM Collins, U Bhatt, A Weller - Proceedings of the AAAI conference on …, 2022 - ojs.aaai.org
The labels used to train machine learning (ML) models are of paramount importance.
Typically for ML classification tasks, datasets contain hard labels, yet learning using soft …

Representation in AI evaluations

AS Bergman, LA Hendricks, M Rauh, B Wu… - Proceedings of the …, 2023 - dl.acm.org
Calls for representation in artificial intelligence (AI) and machine learning (ML) are
widespread, with" representation" or" representativeness" generally understood to be both …

A Data-Centric AI Paradigm for Socio-Industrial and Global Challenges

A Majeed, SO Hwang - Electronics, 2024 - mdpi.com
Due to huge investments by both the public and private sectors, artificial intelligence (AI) has
made tremendous progress in solving multiple real-world problems such as disease …

Url: A representation learning benchmark for transferable uncertainty estimates

M Kirchhof, B Mucsányi, SJ Oh… - Advances in Neural …, 2023 - proceedings.neurips.cc
Abstract Representation learning has significantly driven the field to develop pretrained
models that can act as a valuable starting point when transferring to new datasets. With the …

Probabilistic contrastive learning recovers the correct aleatoric uncertainty of ambiguous inputs

M Kirchhof, E Kasneci, SJ Oh - International Conference on …, 2023 - proceedings.mlr.press
Contrastively trained encoders have recently been proven to invert the data-generating
process: they encode each input, eg, an image, into the true latent vector that generated the …

Conformalized credal set predictors

A Javanmardi, D Stutz… - Advances in Neural …, 2025 - proceedings.neurips.cc
Credal sets are sets of probability distributions that are considered as candidates for an
imprecisely known ground-truth distribution. In machine learning, they have recently …

Codis: Benchmarking context-dependent visual comprehension for multimodal large language models

F Luo, C Chen, Z Wan, Z Kang, Q Yan, Y Li… - arxiv preprint arxiv …, 2024 - arxiv.org
Multimodal large language models (MLLMs) have demonstrated promising results in a
variety of tasks that combine vision and language. As these models become more integral to …