Empowering Self-Balance of Deep Information Bottleneck for Multimodal Clustering

Abstract

Multimodal clustering (MMC) focuses on learning consistent representations through fusing discriminative features from each modality in an unsupervised fashion. Recently, information bottleneck-based MMC methods transform the representation learning into a non-redundant multimodal feature puzzle process. However, they depend on manually setting the trade-off parameter $\beta$ to compress task-irrelevant features while preserving task-relevant features, which severely impairs final clustering results. In addition, manually setting $\beta$ overlooks both the \textit{information quality balance} and the \textit{information quantity balance}, hindering the learning of compact and meaningful representations for discovering cluster patterns. In this work, we propose a novel $\beta_{qq}$-guided information bottleneck ($\beta_{qq}$ IB) for addressing the aforementioned problems. The core of $\beta_{qq}$ IB is a self-balance mechanism that consists of a $\beta_{quality}$ component and a $\beta_{quantity}$ component. The $\beta_{quality}$ component aims to achieve an information quality balance by modeling the relation between task characteristics and data sample scale. Meanwhile, the $\beta_{quantity}$ component intends to realize information quantity balance through modeling the dynamic changes of data scale within the deep variational architect. Experiments on six benchmark datasets demonstrate the superiority of $\beta_{qq}$ IB method. To the best of our knowledge, this is the first work to investigate self-balance learning in IB-based multimodal clustering.