A sparse CCA algorithm with application to model-order selection for small sample support

Abstract

We address the problem of determining the number of signals correlated between two high-dimensional data sets with small sample support. In this setting, conventional techniques based on canonical correlation analysis (CCA) cannot be directly applied since the canonical correlations are significantly overestimated when computed from few samples. To overcome this problem, a principal component analysis (PCA) preprocessing step is usually performed to reduce the dimension of the data. However, PCA reduces the dimension of each data set individually without taking the correlation between the data sets into account. In this paper we propose a sparse CCA (SCCA) algorithm as an alternative to the PCA-CCA approach. This algorithm is based on ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm penalization, which optimizes the weight of the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm to keep a prescribed number of non-zero components. The number of correlated components is then selected based on an information-theoretic criterion.