Anna Rudenko

SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers

The performance of Transformer models has been enhanced by increasing the number of parameters and the length of the processed text. Consequently, fine-tuning the entire model becomes a memory-intensive process. High-performance methods for parameter-efficient fine-tuning (PEFT) typically work with…