Optimization algorithms can see their local convergence rates deteriorate when the Hessian
at the optimum is singular. These singularities are inescapable when the optima are non …

This paper presents a framework for smooth optimization of objectives with ℓq and ℓp, q
regularization for (structured) sparsity. Finding solutions to these non-smooth and possibly …

Gradient descent (GD) is crucial for generalization in machine learning models, as it induces
implicit regularization, promoting compact representations. In this work, we examine the role …

Direct policy search has achieved great empirical success in reinforcement learning. Many
recent studies have revisited its theoretical foundation for continuous control, which reveals …

This paper studies the role of over-parametrization in solving non-convex optimization
problems. The focus is on the important class of low-rank matrix sensing, where we propose …

Homotopy optimization is a traditional method to deal with a complicated optimization
problem by solving a sequence of easy-to-hard surrogate subproblems. However, this …

We study shallow neural networks with polynomial activations. The function space for these
models can be identified with a set of symmetric tensors with bounded rank. We describe …

We consider the problem of minimizing a differentiable function with locally Lipschitz
continuous gradient over the real determinantal variety, and present a first-order algorithm …

The standard simplex in, also known as the probability simplex, is the set of nonnegative
vectors whose entries sum up to 1. It frequently appears as a constraint in optimization …

We consider the problem of minimizing a differentiable function with locally Lipschitz
continuous gradient on a stratified set and present a first-order algorithm designed to find a …