Generalization bounds: Perspectives from information theory and PAC-Bayes

F Hellström, G Durisi, B Guedj… - … and Trends® in …, 2025‏ - nowpublishers.com
A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …

Generalized semi-supervised learning via self-supervised feature adaptation

J Liang, R Hou, H Chang, B Ma… - Advances in Neural …, 2024‏ - proceedings.neurips.cc
Traditional semi-supervised learning (SSL) assumes that the feature distributions of labeled
and unlabeled data are consistent which rarely holds in realistic scenarios. In this paper, we …

Towards generalization beyond pointwise learning: A unified information-theoretic perspective

Y Dong, T Gong, H Chen, Z He, M Li… - Forty-first International …, 2024‏ - openreview.net
The recent surge in contrastive learning has intensified the interest in understanding the
generalization of non-pointwise learning paradigms. While information-theoretic analysis …

Information-theoretic characterizations of generalization error for the Gibbs algorithm

G Aminian, Y Bu, L Toni… - IEEE Transactions …, 2023‏ - ieeexplore.ieee.org
Various approaches have been developed to upper bound the generalization error of a
supervised learning algorithm. However, existing bounds are often loose and even vacuous …

Domain adaptation with domain specific information and feature disentanglement for bearing fault diagnosis

S **e, P **a, H Zhang - Measurement Science and Technology, 2024‏ - iopscience.iop.org
Collecting bearing fault signals from several rotating machines or under varied operating
conditions often results in data distribution offset. Furthermore, the newly obtained data is …

In all likelihoods: Robust selection of pseudo-labeled data

J Rodemann, C Jansen… - International …, 2023‏ - proceedings.mlr.press
Self-training is a simple yet effective method within semi-supervised learning. Self-training's
rationale is to iteratively enhance training data by adding pseudo-labeled data. Its …

Approximately Bayes-optimal pseudo-label selection

J Rodemann, J Goschenhofer… - Uncertainty in …, 2023‏ - proceedings.mlr.press
Semi-supervised learning by self-training heavily relies on pseudo-label selection (PLS).
This selection often depends on the initial model fit on labeled data. Early overfitting might …

Learning under distribution mismatch and model misspecification

MS Masiha, A Gohari, MH Yassaee… - 2021 IEEE International …, 2021‏ - ieeexplore.ieee.org
We study learning algorithms when there is a mismatch between the distributions of the
training and test datasets of a learning algorithm. The effect of this mismatch on the …

Information-theoretic generalization bounds for transductive learning and its applications

H Tang, Y Liu - Journal of Machine Learning Research, 2024‏ - jmlr.org
In this paper, we establish generalization bounds for transductive learning algorithms in the
context of information theory and PAC-Bayes, covering both the random sampling and the …

Information-theoretic characterization of the generalization error for iterative semi-supervised learning

H He, H Yan, VYF Tan - Journal of Machine Learning Research, 2022‏ - jmlr.org
Using information-theoretic principles, we consider the generalization error (gen-error) of
iterative semi-supervised learning (SSL) algorithms that iteratively generate pseudolabels …