← Search

Daesin Kim

2 accepted papers

2022

PAC-Net: A Model Pruning Approach to Inductive Transfer Learning

ICML 2022spotlight

Inductive transfer learning aims to learn from a small amount of training data for the target task by utilizing a pre-trained model from the source task. Most strategies that involve large-scale deep learning models adopt initialization with the pre-trained model and fine-tuning for the target task.…

2021

Learning Student-Friendly Teacher Networks for Knowledge Distillation

NeurIPS 2021poster

We propose a novel knowledge distillation approach to facilitate the transfer of dark knowledge from a teacher to a student. Contrary to most of the existing methods that rely on effective training of student models given pretrained teachers, we aim to learn the teacher models that are friendly to s…

Cited by 119SourcePDFScholar