2026
A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions
ICML 2026poster
Despite the success of knowledge distillation (KD) in Large Language Models (LLMs), the underlying mechanism behind its efficacy remains unclear. In this paper, we propose a unified approach to explore the common mechanism of various KD methods using interactions. Specifically, we decompose the outp…