Foundations & trends in multimodal machine learning: Principles, challenges, and open questions

PP Liang, A Zadeh, LP Morency - ACM Computing Surveys, 2024 - dl.acm.org
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions

PP Liang, A Zadeh, LP Morency - arxiv preprint arxiv:2209.03430, 2022 - arxiv.org
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

Counterfactual attention learning for fine-grained visual categorization and re-identification

Y Rao, G Chen, J Lu, J Zhou - Proceedings of the IEEE/CVF …, 2021 - openaccess.thecvf.com
Attention mechanism has demonstrated great potential in fine-grained visual recognition
tasks. In this paper, we present a counterfactual attention learning method to learn more …

Causal intervention for weakly-supervised semantic segmentation

D Zhang, H Zhang, J Tang… - Advances in Neural …, 2020 - proceedings.neurips.cc
We present a causal inference framework to improve Weakly-Supervised Semantic
Segmentation (WSSS). Specifically, we aim to generate better pixel-level pseudo-masks by …

Unbiased scene graph generation from biased training

K Tang, Y Niu, J Huang, J Shi… - Proceedings of the …, 2020 - openaccess.thecvf.com
Today's scene graph generation (SGG) task is still far from practical, mainly due to the
severe training bias, eg, collapsing diverse" human walk on/sit on/lay on beach" into" human …

Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system

T Wei, F Feng, J Chen, Z Wu, J Yi, X He - Proceedings of the 27th ACM …, 2021 - dl.acm.org
The general aim of the recommender system is to provide personalized suggestions to
users, which is opposed to suggesting popular items. However, the normal training …

A survey on causal inference

L Yao, Z Chu, S Li, Y Li, J Gao, A Zhang - ACM Transactions on …, 2021 - dl.acm.org
Causal inference is a critical research topic across many domains, such as statistics,
computer science, education, public policy, and economics, for decades. Nowadays …

Deciphering spatio-temporal graph forecasting: A causal lens and treatment

Y **a, Y Liang, H Wen, X Liu, K Wang… - Advances in …, 2024 - proceedings.neurips.cc
Abstract Spatio-Temporal Graph (STG) forecasting is a fundamental task in many real-world
applications. Spatio-Temporal Graph Neural Networks have emerged as the most popular …

Counterfactual vqa: A cause-effect look at language bias

Y Niu, K Tang, H Zhang, Z Lu… - Proceedings of the …, 2021 - openaccess.thecvf.com
Recent VQA models may tend to rely on language bias as a shortcut and thus fail to
sufficiently learn the multi-modal knowledge from both vision and language. In this paper …

Cross-modal causal relational reasoning for event-level visual question answering

Y Liu, G Li, L Lin - IEEE Transactions on Pattern Analysis and …, 2023 - ieeexplore.ieee.org
Existing visual question answering methods often suffer from cross-modal spurious
correlations and oversimplified event-level reasoning processes that fail to capture event …