2025
Frozen Language Models Are Gradient Coherence Rectifiers in Vision Transformers
AAAI 2025technical
Large language models (LLMs) have demonstrated remarkable performance in multimodal tasks even with frozen LLM Block and only a few trainable parameters. However, the underlying mechanisms of how LLMs enhance multimodal performance remains unclear. In this work, we focus on the phenomenon that ``Mer…