Uncertainty-Aware Dynamic Fusion for Multimodal Clinical Prediction Tasks

Jiayu Guo; Ying Cheng; Wen He; Yuejie Zhang; Rui Feng; Xiaobo Zhang

ICASSP 2025accepted0 citations

Uncertainty-Aware Dynamic Fusion for Multimodal Clinical Prediction Tasks

Jiayu Guo, Ying Cheng, Wen He, Yuejie Zhang, Rui Feng, Xiaobo Zhang

Abstract

Multimodal fusion offers significant potential for enhancing medical diagnosis, particularly in the Intensive Care Unit (ICU), where integrating diverse data sources is crucial. Traditional static fusion models often fail to account for sample-wise variations in modality importance, which can impact prediction accuracy. To address this issue, we propose a dynamic Uncertainty-Aware Weighting (UAW) strategy that adaptively adjusts the importance of different modalities based on their reliability. This strategy is coupled with an Expert Ensemble Fusion (EEF) module, which leverages self-attention mechanisms and modality-specific FeedForward Networks (FFNs) to preserve and integrate critical information from various modalities. The proposed method demonstrates its efficacy through extensive experiments on phenotype classification and mortality prediction tasks, showing improved accuracy and robustness in handling diverse clinical data.

BibTeX

@inproceedings{icassp2025_uncertaintyaware,
  title = {Uncertainty-Aware Dynamic Fusion for Multimodal Clinical Prediction Tasks},
  author = {Jiayu Guo and Ying Cheng and Wen He and Yuejie Zhang and Rui Feng and Xiaobo Zhang},
  booktitle = {ICASSP 2025},
  year = {2025}
}