Data and its (dis) contents: A survey of dataset development and use in machine learning research

A Paullada, ID Raji, EM Bender, E Denton, A Hanna - Patterns, 2021 - cell.com
In this work, we survey a breadth of literature that has revealed the limitations of
predominant practices for dataset collection and use in the field of machine learning. We …

Counterfactual vqa: A cause-effect look at language bias

Y Niu, K Tang, H Zhang, Z Lu… - Proceedings of the …, 2021 - openaccess.thecvf.com
Recent VQA models may tend to rely on language bias as a shortcut and thus fail to
sufficiently learn the multi-modal knowledge from both vision and language. In this paper …

Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models

T Wu, MT Ribeiro, J Heer, DS Weld - ar** trustworthy ai systems
N Ganguly, D Fazlija, M Badar, M Fisichella… - arxiv preprint arxiv …, 2023 - arxiv.org
State-of-the-art AI models largely lack an understanding of the cause-effect relationship that
governs human understanding of the real world. Consequently, these models do not …

On the value of out-of-distribution testing: An example of goodhart's law

D Teney, E Abbasnejad, K Kafle… - Advances in neural …, 2020 - proceedings.neurips.cc
Abstract Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine
learning system's ability to generalize beyond the biases of a training set. OOD benchmarks …

Evading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization

D Teney, E Abbasnejad, S Lucey… - Proceedings of the …, 2022 - openaccess.thecvf.com
Neural networks trained with SGD were recently shown to rely preferentially on linearly-
predictive features and can ignore complex, equally-predictive ones. This simplicity bias can …

Counterfactual generative networks

A Sauer, A Geiger - arxiv preprint arxiv:2101.06046, 2021 - arxiv.org
Neural networks are prone to learning shortcuts--they often model simple correlations,
ignoring more complex ones that potentially generalize better. Prior works on image …

Learning to contrast the counterfactual samples for robust visual question answering

Z Liang, W Jiang, H Hu, J Zhu - Proceedings of the 2020 …, 2020 - aclanthology.org
In the task of Visual Question Answering (VQA), most state-of-the-art models tend to learn
spurious correlations in the training set and achieve poor performance in out-of-distribution …

Explaining NLP models via minimal contrastive editing (MiCE)

A Ross, A Marasović, ME Peters - arxiv preprint arxiv:2012.13985, 2020 - arxiv.org
Humans have been shown to give contrastive explanations, which explain why an observed
event happened rather than some other counterfactual event (the contrast case). Despite the …

Mutant: A training paradigm for out-of-distribution generalization in visual question answering

T Gokhale, P Banerjee, C Baral, Y Yang - arxiv preprint arxiv:2009.08566, 2020 - arxiv.org
While progress has been made on the visual question answering leaderboards, models
often utilize spurious correlations and priors in datasets under the iid setting. As such …