Deep learning algorithms often require solving a highly non-linear and nonconvex
unconstrained optimization problem. Generally, methods for solving the optimization …

Deep learning algorithms often require solving a highly nonlinear and non-convex
unconstrained optimization problem. Methods for solving optimization problems in large …

While first-order methods are popular for solving optimization problems arising in deep
learning, they come with some acute deficiencies. To overcome these shortcomings, there …

While first-order methods are popular for solving optimization problems that arise in large-
scale deep learning problems, they come with some acute deficiencies. To diminish such …

In this work, we study stochastic quasi-Newton methods for solving the non-linear and non-
convex optimization problems arising in the training of deep neural networks. We consider …

In this paper, we present the compact representation for matrices belonging to the Broyden
class of quasi‐Newton updates, where each update may be either rank one or rank two. This …

Deep learning is a highly effective machine learning technique for large-scale problems.
The optimization of nonconvex functions in deep learning literature is typically restricted to …

Abstract Deep Learning (DL), utilizing Deep Neural Networks (DNNs), has gained significant
popularity in Machine Learning (ML) due to its wide range of applications in various …

Critical to accurate reconstruction of sparse signals from low-dimensional observations is
the solution of nonlinear optimization problems that promote sparse solutions. Sparse signal …