A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression
Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, Ivan Dokmanić, David Belius
Abstract
Existing statistical learning guarantees for general kernel regressors often yield loose bounds when used with finite-rank kernels. Yet, finite-rank kernels naturally appear in a number of machine learning problems, e.g. when fine-tuning a pre-trained deep neural network's last layer to adapt it to a novel task when performing transfer learning. We address this gap for finite-rank kernel ridge regression (KRR) by deriving sharp non-asymptotic upper and lower bounds for the KRR test error of any finite-rank KRR. Our bounds are tighter than previously derived bounds on finite-rank KRR and, unlike comparable results, they also remain valid for any regularization parameters.
BibTeX
@inproceedings{
cheng2023a,
title={A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression},
author={Tin Sum Cheng and Aurelien Lucchi and Anastasis Kratsios and Ivan Dokmani{\'c} and David Belius},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=lk6KDG6qI7}
}