Christof Schüpbach

Fine-Tuning Self-Supervised Models for Language Identification Using Orthonormal Constraint

Self-supervised models trained with high linguistic diversity, such as the XLS-R model, can be effectively fine-tuned for the language recognition task. Typically, a back-end classifier followed by statistics pooling layer are added during training. Commonly used back-end classifiers require a large…