Tao Li*

Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy

"Pre-training followed by fine-tuning is widely adopted among practitioners. The performance can be improved by “model soups” [?] via exploring various hyperparameter configurations. The Learned-Soup, a variant of model soups, significantly improves the performance but suffers from substantial memor…