2024
Distill Vision Transformers to CNNs via Teacher Collaboration
ICASSP 2024accepted
The vision transformer (ViT) has recently emerged as a leading approach in various domains, outperforming other methods. Therefore, it is logical to explore the possibility of transferring the superior knowledge from ViT to more compact and cost-effective convolutional neural networks (CNNs). Howeve…