Barproda Halder

Quantifying Knowledge Distillation using Partial Information Decomposition

Knowledge distillation deploys complex machine learning models in resource-constrained environments by training a smaller student model to emulate internal representations of a complex teacher model. However, the teacher's representations can also encode nuisance or additional information not releva…