Deep neural networks excel in supervised learning tasks but are constrained by the need for
extensive labeled data. Self-supervised learning emerges as a promising alternative …

A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …

Deep neural networks have received dramatic success based on the optimization method of
stochastic gradient descent (SGD). However, it is still not clear how to tune hyper …

Deep learning is usually described as an experiment-driven field under continuous criticizes
of lacking theoretical foundations. This problem has been partially fixed by a large volume of …

We provide new theoretical insights on why over-parametrization is effective in learning
neural networks. For a $ k $ hidden node shallow network with quadratic activation and $ n …

An information-theoretic upper bound on the generalization error of supervised learning
algorithms is derived. The bound is constructed in terms of the mutual information between …

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms
initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli …

The information-theoretic framework of Russo and Zou (2016) and Xu and Raginsky (2017)
provides bounds on the generalization error of a learning algorithm in terms of the mutual …

We study the generalization properties of the popular stochastic optimization method known
as stochastic gradient descent (SGD) for optimizing general non-convex loss functions. Our …

We present a novel set of rigorous and computationally efficient topology-based complexity
notions that exhibit a strong correlation with the generalization gap in modern deep neural …