The overlap gap property: A topological barrier to optimizing over random structures

D Gamarnik - Proceedings of the National Academy of …, 2021 - National Acad Sciences
The problem of optimizing over random structures emerges in many areas of science and
engineering, ranging from statistical physics to machine learning and artificial intelligence …

Frozen 1-RSB structure of the symmetric Ising perceptron

W Perkins, C Xu - Proceedings of the 53rd Annual ACM SIGACT …, 2021 - dl.acm.org
We prove, under an assumption on the critical points of a real-valued function, that the
symmetric Ising perceptron exhibits thefrozen 1-RSB'structure conjectured by Krauth and …

Algorithms and barriers in the symmetric binary perceptron model

D Gamarnik, EC Kızıldağ, W Perkins… - 2022 IEEE 63rd Annual …, 2022 - ieeexplore.ieee.org
The binary (or Ising) perceptron is a toy model of a single-layer neural network and can be
viewed as a random constraint satisfaction problem with a high degree of connectivity. The …

Learning through atypical phase transitions in overparameterized neural networks

C Baldassi, C Lauditi, EM Malatesta, R Pacelli… - Physical Review E, 2022 - APS
Current deep neural networks are highly overparameterized (up to billions of connection
weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient …

[HTML][HTML] Hebbian dreaming for small datasets

E Agliari, F Alemanno, M Aquaro, A Barra, F Durante… - Neural Networks, 2024 - Elsevier
The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for
neural networks, that is able to perform on-line learning when “awake” and also to account …

Proof of the contiguity conjecture and lognormal limit for the symmetric perceptron

E Abbe, S Li, A Sly - 2021 IEEE 62nd Annual Symposium on …, 2022 - ieeexplore.ieee.org
We consider the symmetric binary perceptron model, a simple model of neural networks that
has gathered significant attention in the statistical physics, information theory and probability …

Geometric barriers for stable and online algorithms for discrepancy minimization

D Gamarnik, EC Kizildağ… - The Thirty Sixth Annual …, 2023 - proceedings.mlr.press
For many computational problems involving randomness, intricate geometric features of the
solution space have been used to rigorously rule out powerful classes of algorithms. This is …

On the atypical solutions of the symmetric binary perceptron

D Barbier, A El Alaoui, F Krzakala… - Journal of Physics A …, 2024 - iopscience.iop.org
We study the random binary symmetric perceptron problem, focusing on the behavior of rare
high-margin solutions. While most solutions are isolated, we demonstrate that these rare …

Symmetric perceptron with random labels

EC Kızıldağ, T Wakhare - 2023 International Conference on …, 2023 - ieeexplore.ieee.org
The symmetric binary perceptron (SBP) is a random constraint satisfaction problem (CSP)
and a single-layer neural network; it exhibits intriguing features, most notably a sharp phase …

Solvable model for the linear separability of structured data

M Gherardi - Entropy, 2021 - mdpi.com
Linear separability, a core concept in supervised machine learning, refers to whether the
labels of a data set can be captured by the simplest possible machine: a linear classifier. In …