Unifying Orthogonal Monte Carlo Methods

Krzysztof Choromanski; Mark Rowland; Wenyu Chen; Adrian Weller

ICML 2019oral32 citations

Unifying Orthogonal Monte Carlo Methods

Krzysztof Choromanski, Mark Rowland, Wenyu Chen, Adrian Weller

Abstract

Many machine learning methods making use of Monte Carlo sampling in vector spaces have been shown to be improved by conditioning samples to be mutually orthogonal. Exact orthogonal coupling of samples is computationally intensive, hence approximate methods have been of great interest. In this paper, we present a unifying perspective of many approximate methods by considering Givens transformations, propose new approximate methods based on this framework, and demonstrate the ﬁrst statistical guarantees for families of approximate methods in kernel approximation. We provide extensive empirical evaluations with guidance for practitioners.

BibTeX

@InProceedings{pmlr-v97-choromanski19a,
  title = 	 {Unifying Orthogonal {M}onte {C}arlo Methods},
  author =       {Choromanski, Krzysztof and Rowland, Mark and Chen, Wenyu and Weller, Adrian},
  booktitle = 	 {Proceedings of the 36th International Conference on Machine Learning},
  pages = 	 {1203--1212},
  year = 	 {2019},
  editor = 	 {Chaudhuri, Kamalika and Salakhutdinov, Ruslan},
  volume = 	 {97},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {09--15 Jun},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v97/choromanski19a/choromanski19a.pdf},
  url = 	 {https://proceedings.mlr.press/v97/choromanski19a.html},
  abstract = 	 {Many machine learning methods making use of Monte Carlo sampling in vector spaces have been shown to be improved by conditioning samples to be mutually orthogonal. Exact orthogonal coupling of samples is computationally intensive, hence approximate methods have been of great interest. In this paper, we present a unifying perspective of many approximate methods by considering Givens transformations, propose new approximate methods based on this framework, and demonstrate the ﬁrst statistical guarantees for families of approximate methods in kernel approximation. We provide extensive empirical evaluations with guidance for practitioners.}
}