How does information bottleneck help deep learning?

K Kawaguchi, Z Deng, X Ji… - … Conference on Machine …, 2023 - proceedings.mlr.press
Numerous deep learning algorithms have been inspired by and understood via the notion of
information bottleneck, where unnecessary information is (often implicitly) minimized while …

Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective

Z Yin, E **ng, Z Shen - Advances in Neural Information …, 2024 - proceedings.neurips.cc
We present a new dataset condensation framework termed Squeeze, Recover and Relabel
(SRe $^ 2$ L) that decouples the bilevel optimization of model and synthetic data during …

Mutual Information-Based Generalisation Gap Analysis Using Deep Learning Model

HK Bhuyan, B Unhelkar, S Siva Shankar… - Journal of Information …, 2024 - World Scientific
Most deep learning models face difficulties in analysing image information due to the
concept of information bottlenecks and their corresponding methodologies. But, the …

Towards the Generalization of Multi-view Learning: An Information-theoretical Analysis

W Wen, T Gong, Y Dong, S Yu, W Zhang - arxiv preprint arxiv:2501.16768, 2025 - arxiv.org
Multiview learning has drawn widespread attention for its efficacy in leveraging cross-view
consensus and complementarity information to achieve a comprehensive representation of …

Deep Information Compression for Robust Computer Vision

A Donald Iain Galloway - 2023 - atrium.lib.uoguelph.ca
Abstract Deep Neural Networks (DNNs) are a modeling technique capable of automatically
extracting complex patterns from data, key to the modern practice of Deep Learning (DL). It …

Detecting Shortcuts using Mutual Information

M Adnan, Y Ioannou, K Tsai, A Galloway, H Tizhoosh… - openreview.net
The failure of deep neural networks to generalize to out-of-distribution (OOD) data is a well-
known problem that raises concerns about the deployment of trained networks in safety …