2026
Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding
ICML 2026spotlight
While on-policy distillation offers dense supervision for training small reasoning models, its optimization dynamics in the multimodal domain remain under-explored. In this work, we challenge the standard monolithic view of Vision-Language Model (VLM) distillation by mathematically decomposing the l…