Large Covariance Matrix Estimation for Groups of Highly Correlated Variables via Nonconvex Optimization

Abstract

This paper addresses the problem of covariance matrix estimation in scenarios where the underlying variables can be divided into groups, with variables within each group being highly correlated. Consequently, the covariance matrix displays both sparse and approximately low-rank characteristics due to these highly correlated groups. By appropriately rearranging the variables, the covariance matrix can be transformed into an approximately block diagonal form. In this work, we investigate the estimation of covariance matrices under this structure in high dimensions. We propose a least squares-based covariance estimation method that incorporates a trace norm along with a nonconvex sparsity regularizer to promote both low-rankness and sparsity. Additionally, we introduce a spectral constraint to ensure the positive semi-definiteness of the covariance matrix, even in cases of finite samples, while permitting the integration of prior spectral information. To solve this nonconvex statistical estimation problem, we develop an algorithm based on the majorization-minimization framework, which iteratively solves a convex subproblem. We provide theoretical guarantees that demonstrate the proposed algorithm converges to an estimator achieving the oracle statistical rate under mild technical conditions. Numerical experiments corroborate these theoretical findings.