NeurIPS 2018spotlight59 citations
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
Jiantao Jiao, Weihao Gao, Yanjun Han
Abstract
We analyze the Kozachenko–Leonenko (KL) fixed k-nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance for any fixed k over H\"{o}lder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a recent minimax lower bound over the H\"{o}lder ball, we show that the KL estimator for any fixed k is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter s of the H\"{o}lder ball for $s \in (0,2]$ and arbitrary dimension d, rendering it the first estimator that provably satisfies this property.
BibTeX
@inproceedings{NEURIPS2018_e9fd7c2c,
author = {Jiao, Jiantao and Gao, Weihao and Han, Yanjun},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/e9fd7c2c6623306db59b6aef5c0d5cac-Paper.pdf},
volume = {31},
year = {2018}
}