Xubin Wu

Explicit Spatial Hint and Implicit Logits Relation: Distilling Heterogeneous Knowledge From Vision Transformer to CNN

A lightweight Convolutional Neural Network (CNN) typically requires knowledge transfer from a large powerful network before it is employed in resource-limited edge devices. Vision Transformer (ViT) possesses an unparalleled capability for global modeling but remains largely unexplored in Knowledge D…