In the age of industry 4.0, deep learning has attracted increasing interest for various
research applications. In recent years, deep learning models have been extensively …

Deep learning is transforming most areas of science and technology, including electron
microscopy. This review paper offers a practical perspective aimed at developers with …

The learning rate warmup heuristic achieves remarkable success in stabilizing training,
accelerating convergence and improving generalization for adaptive stochastic optimization …

Realizing quantum speedup for practically relevant, computationally hard problems is a
central challenge in quantum information science. Using Rydberg atom arrays with up to …

Federated learning is a distributed machine learning paradigm in which a large number of
clients coordinate with a central server to learn a model without sharing their own training …

Most popular optimizers for deep learning can be broadly categorized as adaptive methods
(eg~ Adam) and accelerated schemes (eg~ stochastic gradient descent (SGD) with …

Deep metric learning papers from the past four years have consistently claimed great
advances in accuracy, often more than doubling the performance of decade-old methods. In …

In deep learning, different kinds of deep networks typically need different optimizers, which
have to be chosen after multiple trials, making the training process inefficient. To relieve this …

It is not clear yet why ADAM-alike adaptive gradient algorithms suffer from worse
generalization performance than SGD despite their faster training speed. This work aims to …

Abstract Deep Neural Networks (DNNs) are widely regarded as the most effective learning
tool for dealing with large datasets, and they have been successfully used in thousands of …