Adaptive Submanifold-Preserving Sparse Regression for Feature Selection And Multiclass Classification

Abstract

In this paper, we propose a novel embedded feature selection method, which is able to select the informative and discriminative features with the underlying submanifolds of data in intra-class being well preserved so as to improve the classification performance. Specifically, we first impose the l <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</inf> -norm on both loss function and projection matrix with the aim to suppress the influence of noise and achieve the sparse features selection. Then, a retargeted learning technique is introduced into our model to further enhance the discriminant ability of the projection matrix. Moreover, we design an adaptive graph regularizer to fully utilize the intra-class local submanifold information, which can endow the projection matrix with more locality preserving power and prevent overfitting. What is more, our method jointly optimizes problems of the projection learning, target learning and graph learning instead of separately learning them as in most of existing works, which guarantees an overall optimality in algorithmic performance. Experimental results on both synthetic and real-world image data sets demonstrate the superiorities of our method on local submanifold structure exploration and classification task.