Faster cryptonets: Leveraging sparsity for real-world encrypted inference
Homomorphic encryption enables arbitrary computation over data while it remains
encrypted. This privacy-preserving feature is attractive for machine learning, but requires …
encrypted. This privacy-preserving feature is attractive for machine learning, but requires …
The loss surface of deep and wide neural networks
While the optimization problem behind deep neural networks is highly non-convex, it is
frequently observed in practice that training deep networks seems possible without getting …
frequently observed in practice that training deep networks seems possible without getting …
Safetynets: Verifiable execution of deep neural networks on an untrusted cloud
Inference using deep neural networks is often outsourced to the cloud since it is a
computationally demanding task. However, this raises a fundamental issue of trust. How can …
computationally demanding task. However, this raises a fundamental issue of trust. How can …
The mechanism of prediction head in non-contrastive self-supervised learning
The surprising discovery of the BYOL method shows the negative samples can be replaced
by adding the prediction head to the network. It is mysterious why even when there exist …
by adding the prediction head to the network. It is mysterious why even when there exist …
When is a convolutional filter easy to learn?
We analyze the convergence of (stochastic) gradient descent algorithm for learning a
convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does …
convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does …
On connected sublevel sets in deep learning
Q Nguyen - International conference on machine learning, 2019 - proceedings.mlr.press
This paper shows that every sublevel set of the loss function of a class of deep over-
parameterized neural nets with piecewise linear activation functions is connected and …
parameterized neural nets with piecewise linear activation functions is connected and …
Optimization landscape and expressivity of deep CNNs
We analyze the loss landscape and expressiveness of practical deep convolutional neural
networks (CNNs) with shared weights and max pooling layers. We show that such CNNs …
networks (CNNs) with shared weights and max pooling layers. We show that such CNNs …
On the loss landscape of a class of deep neural networks with no bad local valleys
We identify a class of over-parameterized deep neural networks with standard activation
functions and cross-entropy loss which provably have no bad local valley, in the sense that …
functions and cross-entropy loss which provably have no bad local valley, in the sense that …
Adding one neuron can eliminate all bad local minima
One of the main difficulties in analyzing neural networks is the non-convexity of the loss
function which may have many bad local minima. In this paper, we study the landscape of …
function which may have many bad local minima. In this paper, we study the landscape of …
Understanding the loss surface of neural networks for binary classification
It is widely conjectured that training algorithms for neural networks are successful because
all local minima lead to similar performance; for example, see (LeCun et al., 2015; …
all local minima lead to similar performance; for example, see (LeCun et al., 2015; …