The ever-growing artificial intelligence (AI) applications have greatly reshaped our world in
many areas, eg, smart home, computer vision, natural language processing, etc. Behind …

While stochastic gradient descent (SGD) is still the de facto algorithm in deep learning,
adaptive methods like Clipped SGD/Adam have been observed to outperform SGD across …

Federated learning has attracted increasing attention with the emergence of distributed data.
While extensive federated learning algorithms have been proposed for the non-convex …

Adaptive first-order methods in optimization have widespread ML applications due to their
ability to adapt to non-convex landscapes. However, their convergence guarantees are …

We introduce deep equilibrium nets (DEQNs)—a deep learning‐based method to compute
approximate functional rational expectations equilibria of economic models featuring a …

While stochastic gradient descent (SGD) is still the de facto algorithm in deep learning,
adaptive methods like Adam have been observed to outperform SGD across important tasks …

Injecting noise within gradient descent has several desirable features, such as smoothing
and regularizing properties. In this paper, we investigate the effects of injecting noise before …

Gradient-based algorithms are commonly used for training radial basis function neural
network (RBFNN). However, it is still difficult to avoid vanishing gradient to improve the …

Adaptive gradient methods (AGMs) have been widely used to optimize nonconvex problems
in the deep learning area. We identify two aspects of AGMs that can be further improved …

The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has
been actively applied to solve many problems, such as robust dimensionality reduction and …