Multimodal learning with graphs

Y Ektefaie, G Dasoulas, A Noori, M Farhat… - Nature Machine …, 2023‏ - nature.com
Artificial intelligence for graphs has achieved remarkable success in modelling complex
systems, ranging from dynamic networks in biology to interacting particle systems in physics …

A survey of multi-agent deep reinforcement learning with communication

C Zhu, M Dastani, S Wang - Autonomous Agents and Multi-Agent Systems, 2024‏ - Springer
Communication is an effective mechanism for coordinating the behaviors of multiple agents,
broadening their views of the environment, and to support their collaborations. In the field of …

Focal: Contrastive learning for multimodal time-series sensing signals in factorized orthogonal latent space

S Liu, T Kimura, D Liu, R Wang, J Li… - Advances in …, 2023‏ - proceedings.neurips.cc
This paper proposes a novel contrastive learning framework, called FOCAL, for extracting
comprehensive features from multimodal time-series sensing signals through self …

All in one framework for multimodal re-identification in the wild

H Li, M Ye, M Zhang, B Du - Proceedings of the IEEE/CVF …, 2024‏ - openaccess.thecvf.com
Abstract In Re-identification (ReID) recent advancements yield noteworthy progress in both
unimodal and cross-modal retrieval tasks. However the challenge persists in develo** a …

Self-weighted contrastive learning among multiple views for mitigating representation degeneration

J Xu, S Chen, Y Ren, X Shi, H Shen… - Advances in neural …, 2023‏ - proceedings.neurips.cc
Recently, numerous studies have demonstrated the effectiveness of contrastive learning
(CL), which learns feature representations by pulling in positive samples while pushing …

Geometric-inspired graph-based incomplete multi-view clustering

Z Yang, H Zhang, Y Wei, Z Wang, F Nie, D Hu - Pattern Recognition, 2024‏ - Elsevier
Multi-view clustering methods group data into different clusters by discovering the
consensus in heterogeneous sources, which however becomes difficult when partial views …

M3AE: Multimodal representation learning for brain tumor segmentation with missing modalities

H Liu, D Wei, D Lu, J Sun, L Wang… - Proceedings of the AAAI …, 2023‏ - ojs.aaai.org
Multimodal magnetic resonance imaging (MRI) provides complementary information for sub-
region analysis of brain tumors. Plenty of methods have been proposed for automatic brain …

Contextual augmented global contrast for multimodal intent recognition

K Sun, Z **e, M Ye, H Zhang - Proceedings of the IEEE/CVF …, 2024‏ - openaccess.thecvf.com
Multimodal intent recognition (MIR) aims to perceive the human intent polarity via language
visual and acoustic modalities. The inherent intent ambiguity makes it challenging to …

Identifiability results for multimodal contrastive learning

I Daunhawer, A Bizeul, E Palumbo, A Marx… - arxiv preprint arxiv …, 2023‏ - arxiv.org
Contrastive learning is a cornerstone underlying recent progress in multi-view and
multimodal learning, eg, in representation learning with image/caption pairs. While its …

Missmodal: Increasing robustness to missing modality in multimodal sentiment analysis

R Lin, H Hu - Transactions of the Association for Computational …, 2023‏ - direct.mit.edu
When applying multimodal machine learning in downstream inference, both joint and
coordinated multimodal representations rely on the complete presence of modalities as in …