AAAI 2026technical0 citations
Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)
Suwoong Lee, Seungjae Lee, Yunho Jeon, Junmo Kim
Abstract
We propose Tailored ViT Slimming (TVS), a budget-aware multi-dimensional pruning framework for Vision Transformers. TVS injects learnable masks into MHSA and MLP modules and applies adaptive non-convex sparsity regularization to achieve maximal utilization of parameters under strict module-wise budgets. In addition, by retaining scaled masks after pruning, TVS avoids abrupt accuracy drops and provides stable initialization for fine-tuning. On ImageNet-1k with DeiT-S and DeiT-B, TVS consistently outperforms prior ViT compression methods. This result empirically shows that the non-convex sparsity regularizer is effective not only in CNNs but also in ViTs.
BibTeX
@inproceedings{aaai2026_tailoredvitslimm,
title = {Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)},
author = {Suwoong Lee and Seungjae Lee and Yunho Jeon and Junmo Kim},
booktitle = {AAAI 2026},
year = {2026}
}