A high-bias, low-variance introduction to machine learning for physicists

P Mehta, M Bukov, CH Wang, AGR Day, C Richardson… - Physics reports, 2019‏ - Elsevier
Abstract Machine Learning (ML) is one of the most exciting and dynamic areas of modern
research and application. The purpose of this review is to provide an introduction to the core …

Memristive devices for new computing paradigms

IH Im, SJ Kim, HW Jang - Advanced Intelligent Systems, 2020‏ - Wiley Online Library
In complementary metal–oxide–semiconductor (CMOS)‐based von Neumann architectures,
the intrinsic power and speed inefficiencies are worsened by the drastic increase in …

Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry

F Pittorino, A Ferraro, G Perugini… - International …, 2022‏ - proceedings.mlr.press
We systematize the approach to the investigation of deep neural network landscapes by
basing it on the geometry of the space of implemented functions rather than the space of …

Sha** the learning landscape in neural networks around wide flat minima

C Baldassi, F Pittorino… - Proceedings of the …, 2020‏ - National Acad Sciences
Learning in deep neural networks takes place by minimizing a nonconvex high-dimensional
loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process …

Computational roles of intrinsic synaptic dynamics

G Shimizu, K Yoshida, H Kasai, T Toyoizumi - Current opinion in …, 2021‏ - Elsevier
Conventional theories assume that long-term information storage in the brain is
implemented by modifying synaptic efficacy. Recent experimental findings challenge this …

An optimal control approach to deep learning and applications to discrete-weight neural networks

Q Li, S Hao - International Conference on Machine Learning, 2018‏ - proceedings.mlr.press
Deep learning is formulated as a discrete-time optimal control problem. This allows one to
characterize necessary conditions for optimality and develop training algorithms that do not …

Probabilistic binary neural networks

JWT Peters, M Welling - arxiv preprint arxiv:1809.03368, 2018‏ - arxiv.org
Low bit-width weights and activations are an effective way of combating the increasing need
for both memory and compute power of Deep Neural Networks. In this work, we present a …

Mean-field inference methods for neural networks

M Gabrié - Journal of Physics A: Mathematical and Theoretical, 2020‏ - iopscience.iop.org
Abstract Machine learning algorithms relying on deep neural networks recently allowed a
great leap forward in artificial intelligence. Despite the popularity of their applications, the …

Solvable model for inheriting the regularization through knowledge distillation

L Saglietti, L Zdeborová - Mathematical and Scientific …, 2022‏ - proceedings.mlr.press
In recent years the empirical success of transfer learning with neural networks has
stimulated an increasing interest in obtaining a theoretical understanding of its core …

Statistical mechanics of continual learning: Variational principle and mean-field potential

C Li, Z Huang, W Zou, H Huang - Physical Review E, 2023‏ - APS
An obstacle to artificial general intelligence is set by continual learning of multiple tasks of a
different nature. Recently, various heuristic tricks, both from machine learning and from …