Hejung Yang

On Fine-Tuning Pre-Trained Speech Models With EMA-Target Self-Supervised Loss

Representation models pre-trained on self-supervised objectives are often fine-tuned for solving downstream tasks. However, fine-tuning can degrade the general knowledge that was originally built up by the pre-training, which could help prevent the model from overfitting given sparse fine-tuning dat…