Deep learning (DL) training algorithms utilize nondeterminism to improve models' accuracy
and training efficiency. Hence, multiple identical training runs (eg, identical training data …

We present a method to formulate algorithm discovery as program search, and apply it to
discover optimization algorithms for deep neural network training. We leverage efficient …

Dataset distillation aims to learn a small synthetic dataset that preserves most of the
information from the original dataset. Dataset distillation can be formulated as a bi-level …

While deep reinforcement learning (RL) has fueled multiple high-profile successes in
machine learning, it is held back from more widespread adoption by its often poor data …

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent
years. Contrary to conventional approaches to AI where tasks are solved from scratch using …

We present Brax, an open source library for rigid body simulation with a focus on
performance and parallelism on accelerators, written in JAX. We present results on a suite of …

Learning to optimize (L2O) is an emerging approach that leverages machine learning to
develop optimization methods, aiming at reducing the laborious iterations of hand …

The popularity of deep learning has led to the curation of a vast number of massive and
multifarious datasets. Despite having close-to-human performance on individual tasks …

Differentiable simulators promise faster computation time for reinforcement learning by
replacing zeroth-order gradient estimates of a stochastic objective with an estimate based …

Modern machine learning requires system designers to specify aspects of the learning
pipeline, such as losses, architectures, and optimizers. Meta-learning, or learning-to-learn …