Multi-modal learning with missing modality via shared-specific feature modelling

H Wang, Y Chen, C Ma, J Avery… - Proceedings of the …, 2023 - openaccess.thecvf.com
The missing modality issue is critical but non-trivial to be solved by multi-modal models.
Current methods aiming to handle the missing modality problem in multi-modal tasks, either …

Chat with the environment: Interactive multimodal perception using large language models

X Zhao, M Li, C Weber, MB Hafez… - 2023 IEEE/RSJ …, 2023 - ieeexplore.ieee.org
Programming robot behavior in a complex world faces challenges on multiple levels, from
dextrous low-level skills to high-level planning and reasoning. Recent pre-trained Large …

Learnable cross-modal knowledge distillation for multi-modal learning with missing modality

H Wang, C Ma, J Zhang, Y Zhang, J Avery… - … Conference on Medical …, 2023 - Springer
The problem of missing modalities is both critical and non-trivial to be handled in multi-
modal models. It is common for multi-modal tasks that certain modalities contribute more …

COLD fusion: Calibrated and ordinal latent distribution fusion for uncertainty-aware multimodal emotion recognition

MK Tellamekala, S Amiriparian… - … on Pattern Analysis …, 2023 - ieeexplore.ieee.org
Automatically recognising apparent emotions from face and voice is hard, in part because of
various sources of uncertainty, including in the input data and the labels used in a machine …

Confidence-aware multi-modality learning for eye disease screening

K Zou, T Lin, Z Han, M Wang, X Yuan, H Chen… - Medical Image …, 2024 - Elsevier
Multi-modal ophthalmic image classification plays a key role in diagnosing eye diseases, as
it integrates information from different sources to complement their respective performances …

Human–AI collaborative multi-modal multi-rater learning for endometriosis diagnosis

H Wang, D Butler, Y Zhang, J Avery… - Physics in Medicine …, 2024 - iopscience.iop.org
Objective. Endometriosis, affecting about 10% of individuals assigned female at birth, is
challenging to diagnose and manage. Diagnosis typically involves the identification of …

Tackling uncertain correspondences for multi-modal entity alignment

L Chen, Y Sun, S Zhang, Y Ye, W Wu… - The Thirty-eighth Annual …, 2024 - openreview.net
Recently, multi-modal entity alignment has emerged as a pivotal endeavor for the integration
of Multi-Modal Knowledge Graphs (MMKGs) originating from diverse data sources. Existing …

Reducing inference cost of Alzheimer's disease identification using an uncertainty-aware ensemble of uni-modal and multi-modal learners

MT Hagos, KM Curran, B Mac Namee… - Scientific Reports, 2025 - nature.com
While multi-modal deep learning approaches trained using magnetic resonance imaging
(MRI) and fluorodeoxyglucose positron emission tomography (FDG PET) data have shown …

Textualized and feature-based models for compound multimodal emotion recognition in the wild

N Richet, S Belharbi, H Aslam, ME Schadt… - arxiv preprint arxiv …, 2024 - arxiv.org
Systems for multimodal emotion recognition (ER) are commonly trained to extract features
from different modalities (eg, visual, audio, and textual) that are combined to predict …

Uncertainty-Aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data

ES Jeon, MP Buman, P Turaga - IEEE Internet of Things …, 2024 - ieeexplore.ieee.org
In applications involving analysis of the wearable sensor data, machine learning techniques
that use features from the topological data analysis (TDA) have demonstrated remarkable …