Complete dictionary recovery over the sphere I: Overview and the geometric picture

J Sun, Q Qu, J Wright - IEEE Transactions on Information …, 2016 - ieeexplore.ieee.org
We consider the problem of recovering a complete (ie, square and invertible) matrix A 0,
from Y∈ R n× p with Y= A 0 X 0, provided X 0 is sufficiently sparse. This recovery problem is …

A geometric analysis of phase retrieval

J Sun, Q Qu, J Wright - Foundations of Computational Mathematics, 2018 - Springer
Can we recover a complex signal from its Fourier magnitudes? More generally, given a set
of m measurements, y_k=\left| a _k^* x\right| yk= ak∗ x for k= 1, ..., mk= 1,…, m, is it possible …

Feature purification: How adversarial training performs robust deep learning

Z Allen-Zhu, Y Li - 2021 IEEE 62nd Annual Symposium on …, 2022 - ieeexplore.ieee.org
Despite the empirical success of using adversarial training to defend deep learning models
against adversarial perturbations, so far, it still remains rather unclear what the principles are …

Global rates of convergence for nonconvex optimization on manifolds

N Boumal, PA Absil, C Cartis - IMA Journal of Numerical …, 2019 - academic.oup.com
We consider the minimization of a cost function f on a manifold using Riemannian gradient
descent and Riemannian trust regions (RTR). We focus on satisfying necessary optimality …

Solving random quadratic systems of equations is nearly as easy as solving linear systems

Y Chen, E Candes - Advances in Neural Information …, 2015 - proceedings.neurips.cc
This paper is concerned with finding a solution x to a quadratic system of equations yi=|< ai,
x>|^ 2, i= 1, 2,..., m. We prove that it is possible to solve unstructured quadratic systems in n …

Nonconvex phase synchronization

N Boumal - SIAM Journal on Optimization, 2016 - SIAM
We estimate n phases (angles) from noisy pairwise relative phase measurements. The task
is modeled as a nonconvex least-squares optimization problem. It was recently shown that …

[LIBRO][B] Dictionary learning algorithms and applications

B Dumitrescu, P Irofti - 2018 - Springer
This book revolves around the question of designing a matrix D∈ Rm× n called dictionary,
such that to obtain good sparse representations y≈ Dx for a class of signals y∈ Rm given …

Complete dictionary recovery over the sphere II: Recovery by Riemannian trust-region method

J Sun, Q Qu, J Wright - IEEE Transactions on Information …, 2016 - ieeexplore.ieee.org
We consider the problem of recovering a complete (ie, square and invertible) matrix A 0,
from Y∈ R n× p with Y= A 0 X 0, provided X 0 is sufficiently sparse. This recovery problem is …

The power of normalization: Faster evasion of saddle points

KY Levy - arxiv preprint arxiv:1611.04831, 2016 - arxiv.org
A commonly used heuristic in non-convex optimization is Normalized Gradient Descent
(NGD)-a variant of gradient descent in which only the direction of the gradient is taken into …

When are nonconvex problems not scary?

J Sun, Q Qu, J Wright - arxiv preprint arxiv:1510.06096, 2015 - arxiv.org
In this note, we focus on smooth nonconvex optimization problems that obey:(1) all local
minimizers are also global; and (2) around any saddle point or local maximizer, the objective …