A Consolidated Cross-Validation Algorithm for Support Vector Machines via Data Reduction

Boxiang Wang; Yi Yang

NeurIPS 2022accept2 citations

A Consolidated Cross-Validation Algorithm for Support Vector Machines via Data Reduction

Abstract

We propose a consolidated cross-validation (CV) algorithm for training and tuning the support vector machines (SVM) on reproducing kernel Hilbert spaces. Our consolidated CV algorithm utilizes a recently proposed exact leave-one-out formula for the SVM and accelerates the SVM computation via a data reduction strategy. In addition, to compute the SVM with the bias term (intercept), which is not handled by the existing data reduction methods, we propose a novel two-stage consolidated CV algorithm. With numerical studies, we demonstrate that our algorithm is about an order of magnitude faster than the two mainstream SVM solvers, kernlab and LIBSVM, with almost the same accuracy.

Cross-validationData reductionExact leave-one-out lemmaReproducing kernel Hilbert spacesSupport vector machines

BibTeX

@inproceedings{
wang2022a,
title={A Consolidated Cross-Validation Algorithm for Support Vector Machines via Data Reduction},
author={Boxiang Wang and Yi Yang},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
year={2022},
url={https://openreview.net/forum?id=UmDaZksRyk}
}