Shortcut learning in deep neural networks

R Geirhos, JH Jacobsen, C Michaelis… - Nature Machine …, 2020 - nature.com
Deep learning has triggered the current rise of artificial intelligence and is the workhorse of
today's machine intelligence. Numerous success stories have rapidly spread all over …

Deep learning: the good, the bad, and the ugly

T Serre - Annual review of vision science, 2019 - annualreviews.org
Artificial vision has often been described as one of the key remaining challenges to be
solved before machines can act intelligently. Recent developments in a branch of machine …

Last layer re-training is sufficient for robustness to spurious correlations

P Kirichenko, P Izmailov, AG Wilson - arxiv preprint arxiv:2204.02937, 2022 - arxiv.org
Neural network classifiers can largely rely on simple spurious features, such as
backgrounds, to make predictions. However, even in these cases, we show that they still …

On feature learning in the presence of spurious correlations

P Izmailov, P Kirichenko, N Gruver… - Advances in Neural …, 2022 - proceedings.neurips.cc
Deep classifiers are known to rely on spurious features—patterns which are correlated with
the target on the training data but not inherently relevant to the learning problem, such as the …

Deep problems with neural network models of human vision

JS Bowers, G Malhotra, M Dujmović… - Behavioral and Brain …, 2023 - cambridge.org
Deep neural networks (DNNs) have had extraordinary successes in classifying
photographic images of objects and are often described as the best models of biological …

Wilds: A benchmark of in-the-wild distribution shifts

PW Koh, S Sagawa, H Marklund… - International …, 2021 - proceedings.mlr.press
Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild …

Noise or signal: The role of image backgrounds in object recognition

K **ao, L Engstrom, A Ilyas, A Madry - arxiv preprint arxiv:2006.09994, 2020 - arxiv.org
We assess the tendency of state-of-the-art object recognition models to depend on signals
from image backgrounds. We create a toolkit for disentangling foreground and background …

[SÁCH][B] Deep learning for the life sciences: applying deep learning to genomics, microscopy, drug discovery, and more

B Ramsundar, P Eastman, P Walters, V Pande - 2019 - books.google.com
Deep learning has already achieved remarkable results in many fields. Now it's making
waves throughout the sciences broadly and the life sciences in particular. This practical …

Gradient starvation: A learning proclivity in neural networks

M Pezeshki, O Kaba, Y Bengio… - Advances in …, 2021 - proceedings.neurips.cc
We identify and formalize a fundamental gradient descent phenomenon resulting in a
learning proclivity in over-parameterized neural networks. Gradient Starvation arises when …

Why do deep convolutional networks generalize so poorly to small image transformations?

A Azulay, Y Weiss - Journal of Machine Learning Research, 2019 - jmlr.org
Abstract Convolutional Neural Networks (CNNs) are commonly assumed to be invariant to
small image transformations: either because of the convolutional architecture or because …