Data science, and machine learning in particular, is rapidly transforming the scientific and
industrial landscapes. The aerospace industry is poised to capitalize on big data and …

In this paper, we show that although the minimizers of cross-entropy and related
classification losses are off at infinity, network weights learned by gradient flow converge in …

Single-index models are a class of functions given by an unknown univariate``link''function
applied to an unknown one-dimensional projection of the input. These models are …

One of the mysteries in the success of neural networks is randomly initialized first order
methods like gradient descent can achieve zero training loss even though the objective …

Optimal transport distances have found many applications in machine learning for their
capacity to compare non-parametric probability distributions. Yet their algorithmic complexity …

In this paper, we study the implicit regularization of the gradient descent algorithm in
homogeneous neural networks, including fully-connected and convolutional neural …

Policy gradients methods apply to complex, poorly understood, control problems by
performing stochastic gradient descent over a parameterized class of polices. Unfortunately …

The generalization mystery of overparametrized deep nets has motivated efforts to
understand how gradient descent (GD) converges to low-loss solutions that generalize well …

Nonsmooth nonconvex optimization problems broadly emerge in machine learning and
business decision making, whereas two core challenges impede the development of …

We study the implicit regularization imposed by gradient descent for learning multi-layer
homogeneous functions including feed-forward fully connected and convolutional deep …