GCNet: Graph completion network for incomplete multimodal learning in conversation
Conversations have become a critical data format on social media platforms. Understanding
conversation from emotion, content and other aspects also attracts increasing attention from …
conversation from emotion, content and other aspects also attracts increasing attention from …
Understanding multimodal contrastive learning and incorporating unpaired data
Abstract Language-supervised vision models have recently attracted great attention in
computer vision. A common approach to build such models is to use contrastive learning on …
computer vision. A common approach to build such models is to use contrastive learning on …
Data augmentation for audio-visual emotion recognition with an efficient multimodal conditional GAN
Audio-visual emotion recognition is the research of identifying human emotional states by
combining the audio modality and the visual modality simultaneously, which plays an …
combining the audio modality and the visual modality simultaneously, which plays an …
Robust audiovisual emotion recognition: Aligning modalities, capturing temporal information, and handling missing features
Emotion recognition using audiovisual features is a challenging task for human-machine
interaction systems. Under ideal conditions (perfect illumination, clean speech signals, and …
interaction systems. Under ideal conditions (perfect illumination, clean speech signals, and …
SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for incomplete multimodal learning in conversational emotion recognition
Abstract Multimodal Emotion Recognition in Conversations (MERC) aims to classify
utterance emotions using textual, auditory, and visual modal features. Most existing MERC …
utterance emotions using textual, auditory, and visual modal features. Most existing MERC …
A novel transformer autoencoder for multi-modal emotion recognition with incomplete data
Multi-modal signals have become essential data for emotion recognition since they can
represent emotions more comprehensively. However, in real-world environments, it is often …
represent emotions more comprehensively. However, in real-world environments, it is often …
Multiplex graph aggregation and feature refinement for unsupervised incomplete multimodal emotion recognition
Abstract Multimodal Emotion Recognition (MER) involves integrating information of various
modalities, including audio, visual, text and physiological signals, to comprehensively grasp …
modalities, including audio, visual, text and physiological signals, to comprehensively grasp …
Alignment-enhanced interactive fusion model for complete and incomplete multimodal hand gesture recognition
Hand gesture recognition (HGR) based on surface electromyogram (sEMG) and
Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for …
Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for …
Deep multimodal learning with missing modality: A survey
During multimodal model training and testing, certain data modalities may be absent due to
sensor limitations, cost constraints, privacy concerns, or data loss, negatively affecting …
sensor limitations, cost constraints, privacy concerns, or data loss, negatively affecting …
Leveraging Knowledge of Modality Experts for Incomplete Multimodal Learning
Multimodal Emotion Recognition (MER) may encounter incomplete multimodal scenarios
caused by sensor damage or privacy protection in practical applications. Existing incomplete …
caused by sensor damage or privacy protection in practical applications. Existing incomplete …