Energy-based Model Guided Self-Supervised Learning for Speaker Verification
Yaqian Hao, Chenguang Hu, Chong Bian, Junlan Feng, Yingying Gao, Shilei Zhang
Abstract
Self-supervised learning (SSL) has significantly advanced speaker verification, especially in scenarios with limited labeled data. This paper introduces Energy-based Confidence-Aware Distillation (EBCA-DINO), an SSL enhancement for speaker verification that integrates Energy-Based Models (EBMs) into the DINO (Distillation with No Labels) framework. EBMs use energy scores to assess data complexity and uncertainty, guiding label-free self-distillation. The adaptive temperature scaling tailors the learning process to data characteristics, allowing the teacher model to dynamically adjust the student model’s focus based on sample difficulty. This energy-aware distillation optimizes speaker verification performance. Experimental results demonstrate that EBCA-DINO improves speaker verification with relative performance gains of 4.3%, 4.9%, and 8.7% on the Vox1-O, E, and H test trials, respectively.
BibTeX
@inproceedings{icassp2025_energybasedmodel,
title = {Energy-based Model Guided Self-Supervised Learning for Speaker Verification},
author = {Yaqian Hao and Chenguang Hu and Chong Bian and Junlan Feng and Yingying Gao and Shilei Zhang},
booktitle = {ICASSP 2025},
year = {2025}
}