Low-rank Adaptation Method for Respiratory Sound Classification: A necessary road towards Large Models

Abstract

Automatic deep learning-based classification of respiratory sounds is important for the diagnosis of lung diseases. In recent years, many researchers have used pre-trained models to learn more comprehensive features of respiratory sounds. However, as the models become larger, the challenges of long training time and high memory consumption are often faced when fine-tuning the pre-trained model. To alleviate this problem, we apply Low Rank Adaptation (LoRA) to freeze most of the pre-trained model weights and inject the trainable rank decomposition matrices into each layer of the Transformer Encoder. Experiments on the ICBHI 2017 dataset show that the LoRA method reduces the number of trainable parameters by 85M and improves the ICBHI score by 0.94% compared to full fine-tuning. This suggests that the LoRA method has great promise for pre-training respiratory sound classification models. The code is publicly available at https://github.com/deegy666/LoRA-RSC.