Large Covariance Matrix Estimation with Oracle Statistical Rate

Abstract

The ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> penalized covariance estimator has been widely used for estimating large sparse covariance matrices. It was recognized that ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> penalty introduces a non-negligible estimation bias, while a proper utilization of non-convex penalty may lead to an estimator with a refined statistical rate of convergence. In this paper, to eliminate the estimation bias we propose to estimate large sparse covariance matrices using the non-convex penalty. It is a challenging task to analyze the theoretical properties of the resulting covariance estimator because popular iterative algorithms for convex optimization no longer have global convergence guarantees for non-convex optimization. To tackle this issue, an efficient algorithm based on the majorization-minimization (MM) is developed by solving a sequence of convex relaxation subproblems. We prove that the proposed estimator computed exactly by the MM-based algorithm achieves the oracle statistical rate under weak assumptions. Our theoretical findings are corroborated through extensive numerical experiments.