Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

Lin Chen; Sheng Xu

ICLR 2021poster118 citations

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

Abstract

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neural tangent kernel and the Laplace kernel include the same set of functions, when both kernels are restricted to the sphere $\mathbb{S}^{d-1}$. Additionally, we prove that the exponential power kernel with a smaller power (making the kernel less smooth) leads to a larger RKHS, when it is restricted to the sphere $\mathbb{S}^{d-1}$ and when it is defined on the entire $\mathbb{R}^d$.

Neural tangent kernelReproducing kernel Hilbert spaceLaplace kernelSingularity analysis

BibTeX

@inproceedings{
chen2021deep,
title={Deep Neural Tangent Kernel and Laplace Kernel Have the Same {\{}RKHS{\}}},
author={Lin Chen and Sheng Xu},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=vK9WrZ0QYQ}
}