DeiT-LT: Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets
Vision Transformer (ViT) has emerged as a prominent architecture for various computer vision tasks. In ViT we divide the input image into patch tokens and process them through a stack of self-attention blocks. However unlike Convolutional Neural Network (CNN) ViT's simple architecture has no informa…