2025
Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
NeurIPS 2025poster
Knowledge distillation is a promising approach to transfer capabilities from complex teacher models to smaller, resource-efficient student models that can be deployed easily, particularly in task-aware scenarios. However, existing methods of task-aware distillation typically require substantial quan…