Evidential Neural GPLDA: A Novel Approach to Quantify Prediction Uncertainty in Speaker Verification Systems

Abstract

The uncertainty of an automatic speaker verification (ASV) system is typically estimated using its overall accuracy. However it fails to express "when" the system is uncertain in a predictive and case-by-case manner. Also, prior to interpreting each prediction made by ASV systems, there is a need to assess if the system is confident about the prediction, which remains less explored in current research. Given the sense that uncertainty of this notion should be associated with the knowledge level held by the system, we propose an Evidential Neural GPLDA back-end inspired by evidential deep learning. This approach quantifies the uncertainty in each prediction based on the density of the training data supporting that prediction. This is achieved by showing if the input representation is similar to those of the training data samples, resulting in a confidence estimate based on training data only. We find loss functions and out-of-domain samples to train the proposed model such that it parameterizes a sharp Dirichlet distribution as low uncertainty and a flat one as high uncertainty. Experiments show that the proposed novel back-end effectively quantifies uncertainty, providing an estimate that aligns with the error rate.