Foundations & trends in multimodal machine learning: Principles, challenges, and open questions

PP Liang, A Zadeh, LP Morency - ACM Computing Surveys, 2024 - dl.acm.org
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

[HTML][HTML] Multimodal federated learning: A survey

L Che, J Wang, Y Zhou, F Ma - Sensors, 2023 - mdpi.com
Federated learning (FL), which provides a collaborative training scheme for distributed data
sources with privacy concerns, has become a burgeoning and attractive research area. Most …

Quantifying & modeling multimodal interactions: An information decomposition framework

PP Liang, Y Cheng, X Fan, CK Ling… - Advances in …, 2023 - proceedings.neurips.cc
The recent explosion of interest in multimodal applications has resulted in a wide selection
of datasets and methods for representing and integrating information from different …

Learning across diverse biomedical data modalities and cohorts: Challenges and opportunities for innovation

S Rajendran, W Pan, MR Sabuncu, Y Chen, J Zhou… - Patterns, 2024 - cell.com
In healthcare, machine learning (ML) shows significant potential to augment patient care,
improve population health, and streamline healthcare workflows. Realizing its full potential …

Toward explainable affective computing: A review

K Cortiñas-Lorenzo, G Lacey - IEEE Transactions on Neural …, 2023 - ieeexplore.ieee.org
Affective computing has an unprecedented potential to change the way humans interact with
technology. While the last decades have witnessed vast progress in the field, multimodal …

High-modality multimodal transformer: Quantifying modality & interaction heterogeneity for high-modality representation learning

PP Liang, Y Lyu, X Fan, J Tsaw, Y Liu, S Mo… - arxiv preprint arxiv …, 2022 - arxiv.org
Many real-world problems are inherently multimodal, from spoken language, gestures, and
paralinguistics humans use to communicate, to force, proprioception, and visual sensors on …

Multimed: Massively multimodal and multitask medical understanding

S Mo, PP Liang - arxiv preprint arxiv:2408.12682, 2024 - arxiv.org
Biomedical data is inherently multimodal, consisting of electronic health records, medical
imaging, digital pathology, genome sequencing, wearable sensors, and more. The …

Neural mixed effects for nonlinear personalized predictions

T Wörtwein, NB Allen, LB Sheeber… - Proceedings of the 25th …, 2023 - dl.acm.org
Personalized prediction is a machine learning approach that predicts a person's future
observations based on their past labeled observations and is typically used for sequential …

Sequence modeling of passive sensing data for treatment response prediction in major depressive disorder

B Zou, X Zhang, L **ao, R Bai, X Li… - … on Neural Systems …, 2023 - ieeexplore.ieee.org
Major depressive disorder (MDD) is a prevalent mental health condition and has become a
pressing societal challenge. Early prediction of treatment response may aid in the …

Patchwork learning: A paradigm towards integrative analysis across diverse biomedical data sources

S Rajendran, W Pan, MR Sabuncu, Y Chen… - arxiv preprint arxiv …, 2023 - arxiv.org
Machine learning (ML) in healthcare presents numerous opportunities for enhancing patient
care, population health, and healthcare providers' workflows. However, the real-world …