Partial information decomposition for continuous variables based on shared exclusions: Analytical formulation and estimation
Describing statistical dependencies is foundational to empirical scientific research. For
uncovering intricate and possibly nonlinear dependencies between a single target variable …
uncovering intricate and possibly nonlinear dependencies between a single target variable …
Multimodal fusion interactions: A study of human and automatic quantification
In order to perform multimodal fusion of heterogeneous signals, we need to understand their
interactions: how each modality individually provides information useful for a task and how …
interactions: how each modality individually provides information useful for a task and how …
Quantifying spuriousness of biased datasets using partial information decomposition
Spurious patterns refer to a mathematical association between two or more variables in a
dataset that are not causally related. However, this notion of spuriousness, which is usually …
dataset that are not causally related. However, this notion of spuriousness, which is usually …
What should a neuron aim for? Designing local objective functions based on information theory
In modern deep neural networks, the learning dynamics of the individual neurons is often
obscure, as the networks are trained via global optimization. Conversely, biological systems …
obscure, as the networks are trained via global optimization. Conversely, biological systems …
MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts
Advances in multimodal models have greatly improved how interactions relevant to various
tasks are modeled. Today's multimodal models mainly focus on the correspondence …
tasks are modeled. Today's multimodal models mainly focus on the correspondence …
Towards Modality Generalization: A Benchmark and Prospective Analysis
Multi-modal learning has achieved remarkable success by integrating information from
various modalities, achieving superior performance in tasks like recognition and retrieval …
various modalities, achieving superior performance in tasks like recognition and retrieval …
What to align in multimodal contrastive learning?
Humans perceive the world through multisensory integration, blending the information of
different modalities to adapt their behavior. Contrastive learning offers an appealing solution …
different modalities to adapt their behavior. Contrastive learning offers an appealing solution …
Is AI fun? HumorDB: a curated dataset and benchmark to investigate graphical humor
Despite significant advancements in computer vision, understanding complex scenes,
particularly those involving humor, remains a substantial challenge. This paper introduces …
particularly those involving humor, remains a substantial challenge. This paper introduces …
Quantifying Knowledge Distillation Using Partial Information Decomposition
Knowledge distillation provides an effective method for deploying complex machine learning
models in resource-constrained environments. It typically involves training a smaller student …
models in resource-constrained environments. It typically involves training a smaller student …
Mixture of Multimodal Interaction Experts
Multimodal machine learning, which studies the information and interactions across various
input modalities, has made significant advancements in understanding the relationship …
input modalities, has made significant advancements in understanding the relationship …