Enhancing Out-of-Distribution Detection through Dynamic Activation Function

Abstract

In the fields of machine learning and deep learning, ensuring model robustness and reliability is critical. One major challenge is the handling of Out-of-Distribution (OOD) samples, the presence of In-Distribution (ID) noise in existing OOD datasets. It can increase the risk of misclassification and uncertainty in models. Therefore, we first proposed a data purification method driven by multi-model reasoning consistency. As far as we know, this is the first method to automatically purify data in the OOD field. At the same time, we used this method to create the OOD-R (Out-of-Distribution-Rectified) dataset. OOD-R leverages noise filtering methods to improve dataset quality, providing a more reliable benchmark for evaluating OOD detection algorithms. In addition, we propose ActFun, a method designed to improve the model’s response to different inputs, enhance the stability of feature extraction, and mitigate issues associated with model overconfidence. ActFun works by reducing the influence of specific hidden units, enabling the model to better estimate uncertainty in OOD detection and improve generalization ability. Experiments show that implementing ActFun remarkably improves performance on the OOD dataset.