Robust Audio Deepfake Detection using Ensemble Confidence Calibration

Abstract

Model ensembles using linear interpolation are commonly employed to improve classification performance, with higher weights assigned to better-performing models in the ensemble. However, prior methods use fixed weights across all test samples, which is suboptimal as different models may perform better in different subsets of the samples, especially in out-of-domain (OOD) scenarios. This is a key challenge in Audio Deepfake Detection (ADD) due to variations between training and testing domains. To address this, we propose using EOW-Softmax, a method for modeling open-world uncertainties, to calibrate the magnitudes of OOD classification scores at the sample level. This dynamic adjustment improves ensemble predictions on OOD samples. When tested on the ASVspoof 2021 dataset, our calibrated ensemble reduced the equal error rate (EER) from 2.66% to 2.03%.