High-dimensional learning of narrow neural networks

H Cui - arxiv preprint arxiv:2409.13904, 2024 - arxiv.org
Recent years have been marked with the fast-pace diversification and increasing ubiquity of
machine learning applications. Yet, a firm theoretical understanding of the surprising …

High-dimensional learning of narrow neural networks

H Cui - Journal of Statistical Mechanics: Theory and …, 2025 - iopscience.iop.org
Recent years have been marked with the fast-pace diversification and increasing ubiquity of
machine learning (ML) applications. Yet, a firm theoretical understanding of the surprising …

Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects

KL **ao, N Marshall, A Agarwala… - arxiv preprint arxiv …, 2024 - arxiv.org
In recent years, signSGD has garnered interest as both a practical optimizer as well as a
simple model to understand adaptive optimizers like Adam. Though there is a general …