A variety of methods have been applied to the architectural configuration and learning or
training of artificial deep neural networks (DNN). These methods play a crucial role in the …

We propose a meta-learning technique for offline discovery of physics-informed neural
network (PINN) loss functions. We extend earlier works on meta-learning, and develop a …

The evolution of a deep neural network trained by the gradient descent in the
overparametrization regime can be described by its neural tangent kernel (NTK)\cite …

Abstract Graph Neural Networks (GNNs) have been studied through the lens of expressive
power and generalization. However, their optimization properties are less well understood …

A recent line of research on deep learning focuses on the extremely over-parameterized
setting, and shows that when the network width is larger than a high degree polynomial of …

A common method in training neural networks is to initialize all the weights to be
independent Gaussian vectors. We observe that by instead initializing the weights into …

This paper proves that robustness implies generalization via data-dependent generalization
bounds. As a result, robustness and generalization are shown to be connected closely in a …

In these six lectures, we examine what can be learnt about the behavior of multi-layer neural
networks from the analysis of linear models. We first recall the correspondence between …

Overparameterization refers to the important phenomenon where the width of a neural
network is chosen such that learning algorithms can provably attain zero loss in nonconvex …

Abstract In 1988, Eric B. Baum showed that two-layers neural networks with threshold
activation function can perfectly memorize the binary labels of $ n $ points in general …