A new methodology for constructing convex optimization models called disciplined convex
programming is introduced. The methodology enforces a set of conventions upon the …

This paper discusses computational experiments with linear optimization problems involving
semidefinite, quadratic, and linear cone constraints (SQLPs). Many test problems of this type …

We develop exact representations of training two-layer neural networks with rectified linear
units (ReLUs) in terms of a single convex program with number of variables polynomial in …

High-dimension low–sample size statistical analysis is becoming increasingly important in a
wide range of applied contexts. In such situations, the popular support vector machine …

We introduce a computer program PENNON for the solution of problems of convex
Nonlinear and Semidefinite Programming (NLP-SDP). The algorithm used in PENNON is a …

Understanding the fundamental mechanism behind the success of deep neural networks is
one of the key challenges in the modern machine learning literature. Despite numerous …

Pivotal estimation via square-root Lasso in nonparametric regression Page 1 The Annals of
Statistics 2014, Vol. 42, No. 2, 757–788 DOI: 10.1214/14-AOS1204 © Institute of Mathematical …

We develop a convex analytic approach to analyze finite width two-layer ReLU networks.
We first prove that an optimal solution to the regularized training problem can be …

We study training of Convolutional Neural Networks (CNNs) with ReLU activations and
introduce exact convex optimization formulations with a polynomial complexity with respect …

Batch Normalization (BN) is a commonly used technique to accelerate and stabilize training
of deep neural networks. Despite its empirical success, a full theoretical understanding of …