Generalization bounds: Perspectives from information theory and PAC-Bayes

F Hellström, G Durisi, B Guedj… - … and Trends® in …, 2025 - nowpublishers.com
A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …

MMD-FUSE: Learning and combining kernels for two-sample testing without data splitting

F Biggs, A Schrab, A Gretton - Advances in Neural …, 2023 - proceedings.neurips.cc
We propose novel statistics which maximise the power of a two-sample test based on the
Maximum Mean Discrepancy (MMD), byadapting over the set of kernels used in defining it …

Learning via Wasserstein-based high probability generalisation bounds

P Viallard, M Haddouche… - Advances in Neural …, 2023 - proceedings.neurips.cc
Minimising upper bounds on the population risk or the generalisation gap has been widely
used in structural risk minimisation (SRM)--this is in particular at the core of PAC-Bayesian …

Non-vacuous generalisation bounds for shallow neural networks

F Biggs, B Guedj - International Conference on Machine …, 2022 - proceedings.mlr.press
We focus on a specific class of shallow neural networks with a single hidden layer, namely
those with $ L_2 $-normalised data and either a sigmoid-shaped Gaussian error function …

A pac-bayes analysis of adversarial robustness

P Viallard, EG VIDOT, A Habrard… - Advances in Neural …, 2021 - proceedings.neurips.cc
We propose the first general PAC-Bayesian generalization bounds for adversarial
robustness, that estimate, at test time, how much a model will be invariant to imperceptible …

Tighter pac-bayes generalisation bounds by leveraging example difficulty

F Biggs, B Guedj - International Conference on Artificial …, 2023 - proceedings.mlr.press
We introduce a modified version of the excess risk, which can be used to obtain empirically
tighter, faster-rate PAC-Bayesian generalisation bounds. This modified excess risk …

Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets

B Dupuis, P Viallard, G Deligiannidis… - Journal of Machine …, 2024 - jmlr.org
We propose data-dependent uniform generalization bounds by approaching the problem
from a PAC-Bayesian perspective. We first apply the PAC-Bayesian framework on “random …

On margins and generalisation for voting classifiers

F Biggs, V Zantedeschi, B Guedj - Advances in Neural …, 2022 - proceedings.neurips.cc
We study the generalisation properties of majority voting on finite ensembles of classifiers,
proving margin-based generalisation bounds via the PAC-Bayes theory. These provide state …

Shedding a PAC-Bayesian light on adaptive sliced-Wasserstein distances

R Ohana, K Nadjahi… - International …, 2023 - proceedings.mlr.press
Abstract The Sliced-Wasserstein distance (SW) is a computationally efficient and
theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its …

A general framework for the practical disintegration of PAC-Bayesian bounds

P Viallard, P Germain, A Habrard, E Morvant - Machine Learning, 2024 - Springer
PAC-Bayesian bounds are known to be tight and informative when studying the
generalization ability of randomized classifiers. However, they require a loose and costly …