Stochastic gradient descent with a large initial learning rate is widely used for training
modern neural net architectures. Although a small initial learning rate allows for faster …

Existing work on understanding deep learning often employs measures that compress all
data-dependent information into a few numbers. In this work, we adopt a perspective based …

Recent years have witnessed a surge of interest in develo** trustworthy methods to
evaluate the value of data in many real-world applications (eg, collaborative machine …

Learning rate decay (lrDecay) is a\emph {de facto} technique for training modern neural
networks. It starts with a large learning rate and then decays it multiple times. It is empirically …

We study neural network loss landscapes through the lens of mode connectivity, the
observation that minimizers of neural networks retrieved via training on a dataset are …

Inspired by human learning, researchers have proposed ordering examples during training
based on their difficulty. Both curriculum learning, exposing a network to easier examples …

Large web-sourced multimodal datasets have powered a slew of new methods for learning
general-purpose visual representations, advancing the state of the art in computer vision …

Researchers investigating example hardness have increasingly focused on the dynamics by
which neural networks learn and forget examples throughout training. Popular metrics …

In machine learning, a question of great interest is understanding what examples are
challenging for a model to classify. Identifying atypical examples ensures the safe …

Abstract Machine learning (ML) holds great promise for improving healthcare, but it is critical
to ensure that its use will not propagate or amplify health disparities. An important step is to …