Understanding deep learning (still) requires rethinking generalization
Despite their massive size, successful deep artificial neural networks can exhibit a
remarkably small gap between training and test performance. Conventional wisdom …
remarkably small gap between training and test performance. Conventional wisdom …
Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks
Recent works have cast some light on the mystery of why deep nets fit any data and
generalize despite being very overparametrized. This paper analyzes training and …
generalize despite being very overparametrized. This paper analyzes training and …
User-friendly introduction to PAC-Bayes bounds
P Alquier - Foundations and Trends® in Machine Learning, 2024 - nowpublishers.com
Aggregated predictors are obtained by making a set of basic predictors vote according to
some weights, that is, to some probability distribution. Randomized predictors are obtained …
some weights, that is, to some probability distribution. Randomized predictors are obtained …
Generalization bounds: Perspectives from information theory and PAC-Bayes
A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …
decades, the PAC-Bayesian approach has been established as a flexible framework to …
Leveraging unlabeled data to predict out-of-distribution performance
S Garg, S Balakrishnan, ZC Lipton… - ar** non-vacuous generalization bounds for deep
neural networks, these bounds tend to be uninformative about why deep learning works. In …
neural networks, these bounds tend to be uninformative about why deep learning works. In …
Subgroup generalization and fairness of graph neural networks
Despite enormous successful applications of graph neural networks (GNNs), theoretical
understanding of their generalization ability, especially for node-level tasks where data are …
understanding of their generalization ability, especially for node-level tasks where data are …
Tighter risk certificates for neural networks
This paper presents an empirical study regarding training probabilistic neural networks
using training objectives derived from PAC-Bayes bounds. In the context of probabilistic …
using training objectives derived from PAC-Bayes bounds. In the context of probabilistic …
What do compressed deep neural networks forget?
Deep neural network pruning and quantization techniques have demonstrated it is possible
to achieve high levels of compression with surprisingly little degradation to test set accuracy …
to achieve high levels of compression with surprisingly little degradation to test set accuracy …