Optimal Transport with a Diversified Memory Bank for Cross-Domain Speaker Verification
Ruiteng Zhang, Jianguo Wei, Xugang Lu, Wenhuan Lu, Di Jin, Lin Zhang, Junhai Xu
Abstract
Optimal transport (OT) can be applied to cross-domain adaptation in speaker verification (SV) by converting speakers' probability distributions from source to target domains. However, in scenarios involving over-massive categories (speakers) or difficult samples in discrimination, OT often has difficulty computing effective transports. To address this challenge, we propose an OT-based unsupervised domain adaptation (UDA) framework for SV, OT with a diversified memory bank, called DMB-OT, which ensures the accuracy of transfers by two strategies: (1) It regularizes the solution space of OT, which attempts to plan transformations between audio samples from the same speaker with high confidence; (2) it integrates a dynamic curriculum learning algorithm, preventing OT from calculating transport couplings based on hard-discriminative samples in the early stage of UDA. Experiments under different target domains showed that our unsupervised DMB-OT could significantly improve the performance of OT-based UDA and could even match the performance of the supervised PLDA-based adaptation.
BibTeX
@inproceedings{icassp2023_optimaltransport,
title = {Optimal Transport with a Diversified Memory Bank for Cross-Domain Speaker Verification},
author = {Ruiteng Zhang and Jianguo Wei and Xugang Lu and Wenhuan Lu and Di Jin and Lin Zhang and Junhai Xu},
booktitle = {ICASSP 2023},
year = {2023}
}