Sparse Graph Based Sketching for Fast Numerical Linear Algebra

Abstract

In recent years, a variety of randomized constructions of sketching matrices have been devised, that have been used in fast algorithms for numerical linear algebra problems, such as least squares regression, low-rank approximation, and the approximation of leverage scores. A key property of sketching matrices is that of subspace embedding. In this paper, we study sketching matrices that are obtained from bipartite graphs that are sparse, i.e., have left degree s that is small. In particular, we explore two popular classes of sparse graphs, namely, expander graphs and magical graphs. For a given subspace ${\mathcal{U}} \subseteq {{\mathbb{R}}^n}$ of dimension k, we show that the magical graph with left degree s = 2 yields a (1 ± ϵ) ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> -subspace embedding for ${\mathcal{U}}$, if the number of right vertices (the sketch size) $m = {\mathcal{O}}\left( {{k^2}/{\varepsilon ^2}} \right)$. The expander graph with $s = {\mathcal{O}}(\log k/\varepsilon )$ yields a subspace embedding for $m = {\mathcal{O}}\left( {k\log k/{\varepsilon ^2}} \right)$. We also discuss the construction of sparse sketching matrices with reduced randomness using expanders based on error-correcting codes. Empirical results on various synthetic and real datasets show that these sparse graph sketching matrices work very well in practice.