Sparse PCA with Oracle Rate in High Dimensions

Abstract

In this paper, we study the sparse principal component analysis (PCA) problem in high-dimensional settings. We propose a novel row-sparse principal subspace estimator, which estimates the subspace spanned by multiple eigenvectors based on the variance maximization problem with a nonconvex sparsity regularizer. To tackle the nonconvex estimation problem, we introduce a minorization-maximization (MM) algorithm to decompose it into a sequence of convex subproblems. Each subproblem is solved based on the alternating direction method of multipliers. Theoretically, we provide a comprehensive analysis of both the computational and statistical properties of the iterates from the MM algorithm. We demonstrate that the proposed sparse PCA estimator can achieve the same statistical rate as the oracle estimator. Simulation results corroborate the theoretical findings and highlight the superiority of the proposed sparse PCA estimator.