We analyze Newton's method with lazy Hessian updates for solving general possibly non-
convex optimization problems. We propose to reuse a previously seen Hessian for several …

Adaptive optimization methods are widely recognized as among the most popular
approaches for training Deep Neural Networks (DNNs). Techniques such as Adam …

We present a new accelerated stochastic second-order method that is robust to both
gradient and Hessian inexactness, which occurs typically in machine learning. We establish …

In this paper, we propose Cubic Regularized Quasi-Newton Methods for (strongly)
starconvex and Accelerated Cubic Regularized Quasi-Newton for convex optimization. The …

We study the composite convex optimization problems with a Quasi-Self-Concordant smooth
component. This problem class naturally interpolates between classic Self-Concordant …

Second-order methods for convex optimization outperform first-order methods in terms of
theoretical iteration convergence, achieving rates up to $ O (k^{-5}) $ for highly-smooth …

In this paper, we propose the first sketch-and-project Newton method with fast O (k− 2)
global convergence rate for self-concordant functions. Our method, SGN, can be viewed in …

Machine learning assumes a pivotal role in our data-driven world. The increasing scale of
models and datasets necessitates quick and reliable algorithms for model training. This …

The performance of optimization methods is often tied to the spectrum of the objective
Hessian. Yet, conventional assumptions, such as smoothness, do often not enable us to …

Preconditioning is a crucial operation in gradient-based numerical optimisation. It helps
decrease the local condition number of a function by appropriately transforming its gradient …