Foundations & trends in multimodal machine learning: Principles, challenges, and open questions

PP Liang, A Zadeh, LP Morency - ACM Computing Surveys, 2024 - dl.acm.org
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

To compress or not to compress—self-supervised learning and information theory: A review

R Shwartz Ziv, Y LeCun - Entropy, 2024 - mdpi.com
Deep neural networks excel in supervised learning tasks but are constrained by the need for
extensive labeled data. Self-supervised learning emerges as a promising alternative …

Dual contrastive prediction for incomplete multi-view representation learning

Y Lin, Y Gou, X Liu, J Bai, J Lv… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
In this article, we propose a unified framework to solve the following two challenging
problems in incomplete multi-view representation learning: i) how to learn a consistent …

Completer: Incomplete multi-view clustering via contrastive prediction

Y Lin, Y Gou, Z Liu, B Li, J Lv… - Proceedings of the IEEE …, 2021 - openaccess.thecvf.com
In this paper, we study two challenging problems in incomplete multi-view clustering
analysis, namely, i) how to learn an informative and consistent representation among …

From canonical correlation analysis to self-supervised graph neural networks

H Zhang, Q Wu, J Yan, D Wipf… - Advances in Neural …, 2021 - proceedings.neurips.cc
We introduce a conceptually simple yet effective model for self-supervised representation
learning with graph data. It follows the previous methods that generate two views of an input …

Learning representations by maximizing mutual information across views

P Bachman, RD Hjelm… - Advances in neural …, 2019 - proceedings.neurips.cc
We propose an approach to self-supervised representation learning based on maximizing
mutual information between features extracted from multiple views of a shared context. For …

Factorized contrastive learning: Going beyond multi-view redundancy

PP Liang, Z Deng, MQ Ma, JY Zou… - Advances in …, 2023 - proceedings.neurips.cc
In a wide range of multimodal tasks, contrastive learning has become a particularly
appealing approach since it can successfully learn representations from abundant …

Multi-view learning overview: Recent progress and new challenges

J Zhao, X **e, X Xu, S Sun - Information Fusion, 2017 - Elsevier
Multi-view learning is an emerging direction in machine learning which considers learning
with multiple views to improve the generalization performance. Multi-view learning is also …

Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks

N Wu, S Jastrzebski, K Cho… - … Conference on Machine …, 2022 - proceedings.mlr.press
We hypothesize that due to the greedy nature of learning in multi-modal deep neural
networks, these models tend to rely on just one modality while under-fitting the other …

What's behind the mask: Understanding masked graph modeling for graph autoencoders

J Li, R Wu, W Sun, L Chen, S Tian, L Zhu… - Proceedings of the 29th …, 2023 - dl.acm.org
The last years have witnessed the emergence of a promising self-supervised learning
strategy, referred to as masked autoencoding. However, there is a lack of theoretical …