We formally study how ensemble of deep learning models can improve test accuracy, and
how the superior performance of ensemble can be distilled into a single model using …

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
Page 1 Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two …

How can neural networks such as ResNet\emph {efficiently} learn CIFAR-10 with test
accuracy more than $96\% $, while other methods, especially kernel methods, fall relatively …

Despite the empirical success of using adversarial training to defend deep learning models
against adversarial perturbations, so far, it still remains rather unclear what the principles are …

Federated Learning (FL) is an emerging learning scheme that allows different distributed
clients to train deep neural networks together without data sharing. Neural networks have …

This paper provides a theoretical understanding of deep Q-Network (DQN) with the
$\varepsilon $-greedy exploration in deep reinforcement learning. Despite the tremendous …

We consider the dynamic of gradient descent for learning a two-layer neural network. We
assume the input $ x\in\mathbb {R}^ d $ is drawn from a Gaussian distribution and the label …

A common method in training neural networks is to initialize all the weights to be
independent Gaussian vectors. We observe that by instead initializing the weights into …

We give superpolynomial statistical query (SQ) lower bounds for learning two-hidden-layer
ReLU networks with respect to Gaussian inputs in the standard (noise-free) model. No …