Provable repair of deep neural networks
Deep Neural Networks (DNNs) have grown in popularity over the past decade and are now
being used in safety-critical domains such as aircraft collision avoidance. This has motivated …
being used in safety-critical domains such as aircraft collision avoidance. This has motivated …
Learning parities with neural networks
In recent years we see a rapidly growing line of research which shows learnability of various
models via common neural network algorithms. Yet, besides a very few outliers, these …
models via common neural network algorithms. Yet, besides a very few outliers, these …
Unraveling attention via convex duality: Analysis and interpretations of vision transformers
Vision transformers using self-attention or its proposed alternatives have demonstrated
promising results in many image related tasks. However, the underpinning inductive bias of …
promising results in many image related tasks. However, the underpinning inductive bias of …
Fast convex optimization for two-layer relu networks: Equivalent model classes and cone decompositions
We develop fast algorithms and robust software for convex optimization of two-layer neural
networks with ReLU activation functions. Our work leverages a convex re-formulation of the …
networks with ReLU activation functions. Our work leverages a convex re-formulation of the …
The influence of learning rule on representation dynamics in wide neural networks
It is unclear how changing the learning rule of a deep neural network alters its learning
dynamics and representations. To gain insight into the relationship between learned …
dynamics and representations. To gain insight into the relationship between learned …
How does a kernel based on gradients of infinite-width neural networks come to be widely used: a review of the neural tangent kernel
Y Tan, H Liu - International Journal of Multimedia Information …, 2024 - Springer
The neural tangent kernel (NTK) was created in the context of using the limit idea to study
the theory of neural network. NTKs are defined from neural network models in the infinite …
the theory of neural network. NTKs are defined from neural network models in the infinite …
Globally gated deep linear networks
Abstract Recently proposed Gated Linear Networks (GLNs) present a tractable nonlinear
network architecture, and exhibit interesting capabilities such as learning with local error …
network architecture, and exhibit interesting capabilities such as learning with local error …
Optimal sets and solution paths of ReLU networks
We develop an analytical framework to characterize the set of optimal ReLU neural networks
by reformulating the non-convex training problem as a convex program. We show that the …
by reformulating the non-convex training problem as a convex program. We show that the …
[PDF][PDF] Correcting deep neural networks with small, generalizing patches
We consider the problem of patching a deep neural network: applying a small change to the
network weights in order to produce a desired change in the classifications made by the …
network weights in order to produce a desired change in the classifications made by the …
Towards understanding learning in neural networks with linear teachers
Can a neural network minimizing cross-entropy learn linearly separable data? Despite
progress in the theory of deep learning, this question remains unsolved. Here we prove that …
progress in the theory of deep learning, this question remains unsolved. Here we prove that …