On the need for a language describing distribution shifts: Illustrations on tabular datasets
Different distribution shifts require different algorithmic and operational interventions.
Methodological research must be grounded by the specific shifts they address. Although …
Methodological research must be grounded by the specific shifts they address. Although …
Demographic bias in misdiagnosis by computational pathology models
Despite increasing numbers of regulatory approvals, deep learning-based computational
pathology systems often overlook the impact of demographic factors on performance …
pathology systems often overlook the impact of demographic factors on performance …
A survey on evaluation of out-of-distribution generalization
Machine learning models, while progressively advanced, rely heavily on the IID assumption,
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …
The data addition dilemma
In many machine learning for healthcare tasks, standard datasets are constructed by
amassing data across many, often fundamentally dissimilar, sources. But when does adding …
amassing data across many, often fundamentally dissimilar, sources. But when does adding …
Doubly robust augmented model accuracy transfer inference with high dimensional features
Transfer learning is crucial for training models that generalize to unlabeled target
populations using labeled source data, especially in real-world studies where label scarcity …
populations using labeled source data, especially in real-world studies where label scarcity …
AdaptSel: Adaptive Selection of Biased and Debiased Recommendation Models for Varying Test Environments
Recommendation systems are frequently challenged by pervasive biases in the training set
that can compromise model effectiveness. To address this issue, various debiasing …
that can compromise model effectiveness. To address this issue, various debiasing …
Understanding Disparities in Post Hoc Machine Learning Explanation
Previous work has highlighted that existing post-hoc explanation methods exhibit disparities
in explanation fidelity (across “race” and “gender” as sensitive attributes), and while a large …
in explanation fidelity (across “race” and “gender” as sensitive attributes), and while a large …
A Baseline Analysis of Reward Models' Ability To Accurately Analyze Foundation Models Under Distribution Shift
Foundation models, specifically Large Language Models (LLMs), have lately gained wide-
spread attention and adoption. Reinforcement Learning with Human Feedback (RLHF) …
spread attention and adoption. Reinforcement Learning with Human Feedback (RLHF) …
Understanding subgroup performance differences of fair predictors using causal models
A common evaluation paradigm compares the performance of a machine learning model
across subgroups to assess properties related to fairness. In this work, we argue that …
across subgroups to assess properties related to fairness. In this work, we argue that …
Not all distributional shifts are equal: Fine-grained robust conformal inference
We introduce a fine-grained framework for uncertainty quantification of predictive models
under distributional shifts. This framework distinguishes the shift in covariate distributions …
under distributional shifts. This framework distinguishes the shift in covariate distributions …