Piyush Raikwar

Discovering and Overcoming Limitations of Noise-engineered Data-free Knowledge Distillation

Distillation in neural networks using only the samples randomly drawn from a Gaussian distribution is possibly the most straightforward solution one can think of for the complex problem of knowledge transfer from one network (teacher) to the other (student). If successfully done, it can eliminate th…