Uniform Convergence of Lipschitz Functions with Dependent Gaussian Samples

Abstract

In many practical learning problems, training samples are not i.i.d., and there is an intrinsic dependency among samples. Therefore, theoretical study of learning with dependent data has recently gained attention. In this paper, we provide a uniform convergence bound for the class of Lipschitz functions with bounded values at zero, under the assumption that the data samples are scalar and have a possibly dependent joint Gaussian distribution. Since other than Lipschitzness, there is no heavy assumption such as convexity or boundedness on the function class, the results are applicable for many practical models including neural networks. We showcase the strength and applicability of our theorems by numerical simulation and real-data analysis.