Re-Thinking Classification Confidence with Model Quality Quantification

Abstract

Deep neural networks using for real-world classification task require high reliability and robustness. However, the Softmax output by the last layer of network is often over-confident. We propose a novel confidence estimation method by considering model quality for deep classification models. Two metrics, MQ-Repres and MQ-Discri are developed accordingly to evaluate the model quality, and also provide a new confidence estimation called MQ-Conf for online inference. We demonstrate the capability of the proposed method by the 3D3D semantic segmentation tasks using three different deep networks. Through confusion analysis and feature visualization we show the rationality and reliability of the model quality quantification method.11This work is supported by the National Natural Science Foundation of China under Grant U22A2061 and High-performance Computing Platform of Peking University.