The lottery ticket hypothesis (Frankle and Carbin, 2018), states that a randomly-initialized
network contains a small subnetwork such that, when trained in isolation, can compete with …

We give the first statistical-query lower bounds for agnostically learning any non-polynomial
activation with respect to Gaussian marginals (eg, ReLU, sigmoid, sign). For the specific …

This paper introduces a novel physics-informed multi-agents constitutive model to propose
prediction in quasi-static constitutive behavior of cross-linked elastomer and the loss of …

Overwhelming theoretical and empirical evidence shows that mildly overparametrized
neural networks---those with more connections than the size of the training data---are often …

Hybrid private inference (PI) protocol, which synergistically utilizes both multi-party
computation (MPC) and homomorphic encryption, is one of the most prominent techniques …

Incorporating a so-called “momentum” dynamic in gradient descent methods is widely used
in neural net training as it has been broadly observed that, at least empirically, it often leads …

We study the extent to which wide neural networks may be approximated by Gaussian
processes, when initialized with random weights. It is a well-established fact that as the …

In this paper, we propose a novel activation function tailored specifically for graph data in
Graph Neural Networks (GNNs). Motivated by the need for graph-adaptive and flexible …

The nonlinearity of activation functions used in deep learning models is crucial for the
success of predictive models. Several simple nonlinear functions, including Rectified Linear …

Under mild conditions on the network initialization we derive a power series expansion for
the Neural Tangent Kernel (NTK) of arbitrarily deep feedforward networks in the infinite …