Classification via Subspace Learning Machine (SLM): Methodology and Performance Evaluation

Abstract

Inspired by the decision learning process of multilayer per-ceptron (MLP) and decision tree (DT), a new classification model, named the subspace learning machine (SLM), is proposed in this work. SLM first identifies a discriminant subspace, S<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup>, by examining the discriminant power of each input feature. Then, it learns projections of features in S<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> to yield 1D subspaces and finds the optimal partition for each. A criterion is developed to choose the best q partitions that yield 2<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</sup> partitioned subspaces. The partitioning process is recursively applied at each child node to build an SLM tree. When the samples at a child node are sufficiently pure, the partitioning process stops, and each leaf node makes a prediction. The ensembles of SLM trees can yield a stronger predictor. Extensive experiments are conducted for performance benchmarking among SLM trees, ensembles and classical classifiers.