The paper reviews and extends an emerging body of theoretical results on deep learning
including the conditions under which it can be exponentially better than shallow learning. A …

While deep learning is successful in a number of applications, it is not yet well understood
theoretically. A theoretical characterization of deep learning should answer questions about …

The paper briefly reviews several recent results on hierarchical architectures for learning
from examples, that may formally explain the conditions under which Deep Convolutional …

While the universal approximation property holds both for hierarchical and shallow
networks, deep networks can approximate the class of compositional functions as well as …

Deep neural networks (DNNs) trained with the logistic loss (also known as the cross entropy
loss) have made impressive advancements in various binary classification tasks. Despite the …

Convolutional rectifier networks, ie convolutional neural networks with rectified linear
activation and max or average pooling, are the cornerstone of modern deep learning …

While the universal approximation property holds both for hierarchical and shallow
networks, we prove that deep (hierarchical) networks can approximate the class of …

Our formal understanding of the inductive bias that drives the success of convolutional
networks on computer vision tasks is limited. In particular, it is unclear what makes …

Predictive coding has been proposed as a model of the hierarchical perceptual inference
process performed in the cortex. However, results demonstrating that predictive coding is …

We study the approximation of two-layer compositions f (x)= g (ϕ (x)) via deep networks with
ReLU activation, where ϕ is a geometrically intuitive, dimensionality reducing feature map …