On Empirical Risk Minimization with Dependent and Heavy-Tailed Data
Abhishek Roy, Krishna Balasubramanian, Murat A Erdogdu
Abstract
In this work, we establish risk bounds for Empirical Risk Minimization (ERM) with both dependent and heavy-tailed data-generating processes. We do so by extending the seminal works~\cite{pmlr-v35-mendelson14, mendelson2018learning} on the analysis of ERM with heavy-tailed but independent and identically distributed observations, to the strictly stationary exponentially $\beta$-mixing case. We allow for the interaction between the noise and inputs to be even polynomially heavy-tailed, which covers a significantly large class of heavy-tailed models beyond what is analyzed in the learning theory literature. We illustrate our theoretical results by obtaining rates of convergence for high-dimensional linear regression with dependent and heavy-tailed data.
BibTeX
@inproceedings{
roy2021on,
title={On Empirical Risk Minimization with Dependent and Heavy-Tailed Data},
author={Abhishek Roy and Krishna Balasubramanian and Murat A Erdogdu},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=Tzkev89HeLZ}
}