Mert Bulent Sariyildiz*

UNIC: Universal Classification Models via Multi-teacher Distillation

"Pretrained models have become a commodity and offer strong results on a broad range of tasks. In this work, we focus on classification and seek to learn a unique encoder able to take from several complementary pretrained models. We aim at even stronger generalization across a variety of classificat…