Beyond Fixed Biases: Decoding the Role of Reasoning Uncertainty in MLLM Modality Conflicts

Abstract

Multimodal Large Language Models (MLLMs) must resolve conflicts when modalities provide contradictory information, a process we term "modality following". We propose a framework that deconstructs this behavior into case-specific relative reasoning uncertainty and a model's stable inherent preference. By evaluating diverse MLLMs, we establish a universal law: the probability of following a modality decreases monotonically as its relative reasoning uncertainty increases, which is robustly preserved across diverse uncertainty indices. This law allows us to quantify a "balance point" where uncertainties are subjectiveized, offering a principled measure of modality bias that is disentangled from unimodal capabilities. Probing the internal decision-making reveals that this conflict resolution is a high-level cognitive process: in ambiguous regions near the balance point, models exhibit significant "concept oscillations," where top predictions vacillate between modalities specifically within the middle-to-late layers. Finally, we demonstrate the framework's utility for preference steering through Supervised Fine-Tuning (SFT). We find that data efficiency is governed by reasoning uncertainty: training on easy samples (where one modality dominates) fails to generalize, whereas targeting the identified ``boundary cases" is essential for robust preference alignment and suppressing internal vacillation.