← Search

Nicholas Allen

2 accepted papers

2023

Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework

NeurIPS 2023poster

The recent explosion of interest in multimodal applications has resulted in a wide selection of datasets and methods for representing and integrating information from different modalities. Despite these empirical advances, there remain fundamental research questions: How can we quantify the interact…

2022

Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions

EMNLP 2022finding

Multimodal fusion addresses the problem of analyzing spoken words in the multimodal context, including visual expressions and prosodic cues. Even when multimodal models lead to performance improvements, it is often unclear whether bimodal and trimodal interactions are learned or whether modalities a…

Cited by 9SourcePDFScholar