There is mounting evidence of emergent phenomena in the capabilities of deep learning
methods as we scale up datasets, model sizes, and training times. While there are some …

We describe a general technique that yields the first Statistical Query lower bounds for a
range of fundamental high-dimensional learning problems involving Gaussian distributions …

Machine learning is being increasingly used by individuals, research institutions, and
corporations. This has resulted in the surge of Machine Learning-as-a-Service (MLaaS) …

We study the supervised learning problem under either of the following two models:(1)
Feature vectors xi are d-dimensional Gaussian and responses are yi= f*(xi) for f* an …

We study the task of agnostically learning halfspaces under the Gaussian distribution.
Specifically, given labeled examples $(\\mathbf {x}, y) $ from an unknown distribution on …

We give the first dimension-efficient algorithms for learning Rectified Linear Units (ReLUs),
which are functions of the form $\mathbf {x}\mapsto\mathsf {max}(0,\mathbf {w}⋅\mathbf {x}) …

We give the first superpolynomial lower bounds for learning one-layer neural networks with
respect to the Gaussian distribution for a broad class of algorithms. In the regression setting …

We study the fundamental problems of agnostically learning halfspaces and ReLUs under
Gaussian marginals. In the former problem, given labeled examples $(\bx, y) $ from an …

We give superpolynomial statistical query (SQ) lower bounds for learning two-hidden-layer
ReLU networks with respect to Gaussian inputs in the standard (noise-free) model. No …

Although neural networks are routinely and successfully trained in practice using simple
gradient-based methods, most existing theoretical results are negative, showing that …