Jeffrey Cohn

Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions

Multimodal fusion addresses the problem of analyzing spoken words in the multimodal context, including visual expressions and prosodic cues. Even when multimodal models lead to performance improvements, it is often unclear whether bimodal and trimodal interactions are learned or whether modalities a…