V-PETL Bench: A Unified Visual Parameter-Efficient Transfer Learning Benchmark
Yi Xin, Siqi Luo, Xuyang Liu, Yuntao Du., Haodi Zhou, Xinyu Cheng, Christina Luoluo Lee, Junlong Du
Abstract
Parameter-efficient transfer learning (PETL) methods show promise in adapting a pre-trained model to various downstream tasks while training only a few parameters. In the computer vision (CV) domain, numerous PETL algorithms have been proposed, but their direct employment or comparison remains inconvenient. To address this challenge, we construct a Unified Visual PETL Benchmark (V-PETL Bench) for the CV domain by selecting 30 diverse, challenging, and comprehensive datasets from image recognition, video action recognition, and dense prediction tasks. On these datasets, we systematically evaluate 25 dominant PETL algorithms and open-source a modular and extensible codebase for fair evaluation of these algorithms. V-PETL Bench runs on NVIDIA A800 GPUs and requires approximately 310 GPU days. We release all the benchmark, making it more efficient and friendly to researchers. Additionally, V-PETL Bench will be continuously updated for new PETL algorithms and CV tasks.
BibTeX
@inproceedings{
xin2024vpetl,
title={V-{PETL} Bench: A Unified Visual Parameter-Efficient Transfer Learning Benchmark},
author={Yi Xin and Siqi Luo and Xuyang Liu and Yuntao Du. and Haodi Zhou and Xinyu Cheng and Christina Luoluo Lee and Junlong Du and Haozhe Wang and MingCai Chen and Ting Liu and Guimin Hu and Zhongwei Wan and Rongchao Zhang and Aoxue Li and Mingyang Yi and Xiaohong Liu},
booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2024},
url={https://openreview.net/forum?id=yS1dUkQFnu}
}