The overlap gap property: A topological barrier to optimizing over random structures
D Gamarnik - Proceedings of the National Academy of …, 2021 - National Acad Sciences
The problem of optimizing over random structures emerges in many areas of science and
engineering, ranging from statistical physics to machine learning and artificial intelligence …
engineering, ranging from statistical physics to machine learning and artificial intelligence …
Frozen 1-RSB structure of the symmetric Ising perceptron
We prove, under an assumption on the critical points of a real-valued function, that the
symmetric Ising perceptron exhibits thefrozen 1-RSB'structure conjectured by Krauth and …
symmetric Ising perceptron exhibits thefrozen 1-RSB'structure conjectured by Krauth and …
Algorithms and barriers in the symmetric binary perceptron model
D Gamarnik, EC Kızıldağ, W Perkins… - 2022 IEEE 63rd Annual …, 2022 - ieeexplore.ieee.org
The binary (or Ising) perceptron is a toy model of a single-layer neural network and can be
viewed as a random constraint satisfaction problem with a high degree of connectivity. The …
viewed as a random constraint satisfaction problem with a high degree of connectivity. The …
Learning through atypical phase transitions in overparameterized neural networks
Current deep neural networks are highly overparameterized (up to billions of connection
weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient …
weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient …
[HTML][HTML] Hebbian dreaming for small datasets
The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for
neural networks, that is able to perform on-line learning when “awake” and also to account …
neural networks, that is able to perform on-line learning when “awake” and also to account …
Proof of the contiguity conjecture and lognormal limit for the symmetric perceptron
We consider the symmetric binary perceptron model, a simple model of neural networks that
has gathered significant attention in the statistical physics, information theory and probability …
has gathered significant attention in the statistical physics, information theory and probability …
Geometric barriers for stable and online algorithms for discrepancy minimization
D Gamarnik, EC Kizildağ… - The Thirty Sixth Annual …, 2023 - proceedings.mlr.press
For many computational problems involving randomness, intricate geometric features of the
solution space have been used to rigorously rule out powerful classes of algorithms. This is …
solution space have been used to rigorously rule out powerful classes of algorithms. This is …
On the atypical solutions of the symmetric binary perceptron
We study the random binary symmetric perceptron problem, focusing on the behavior of rare
high-margin solutions. While most solutions are isolated, we demonstrate that these rare …
high-margin solutions. While most solutions are isolated, we demonstrate that these rare …
Symmetric perceptron with random labels
EC Kızıldağ, T Wakhare - 2023 International Conference on …, 2023 - ieeexplore.ieee.org
The symmetric binary perceptron (SBP) is a random constraint satisfaction problem (CSP)
and a single-layer neural network; it exhibits intriguing features, most notably a sharp phase …
and a single-layer neural network; it exhibits intriguing features, most notably a sharp phase …
Solvable model for the linear separability of structured data
M Gherardi - Entropy, 2021 - mdpi.com
Linear separability, a core concept in supervised machine learning, refers to whether the
labels of a data set can be captured by the simplest possible machine: a linear classifier. In …
labels of a data set can be captured by the simplest possible machine: a linear classifier. In …