Deep Spectral Ranking
Ilkay Yildiz, Jennifer Dy, Deniz Erdogmus, Susan Ostmo, J. Peter Campbell, Michael F. Chiang, Stratis Ioannidis
Abstract
Learning from ranking observations arises in many domains, and siamese deep neural networks have shown excellent inference performance in this setting. However, SGD does not scale well, as an epoch grows exponentially with the ranking observation size. We show that a spectral algorithm can be combined with deep learning methods to significantly accelerate training. We combine a spectral estimate of Plackett-Luce ranking scores with a deep model via the Alternating Directions Method of Multipliers with a Kullback-Leibler proximal penalty. Compared to a state-of-the-art siamese network, our algorithms are up to 175 times faster and attain better predictions by up to 26% Top-1 Accuracy and 6% Kendall-Tau correlation over five real-life ranking datasets.
BibTeX
@InProceedings{pmlr-v130-yildiz21a,
title = { Deep Spectral Ranking },
author = {Yildiz, Ilkay and Dy, Jennifer and Erdogmus, Deniz and Ostmo, Susan and Peter Campbell, J. and F. Chiang, Michael and Ioannidis, Stratis},
booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
pages = {361--369},
year = {2021},
editor = {Banerjee, Arindam and Fukumizu, Kenji},
volume = {130},
series = {Proceedings of Machine Learning Research},
month = {13--15 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v130/yildiz21a/yildiz21a.pdf},
url = {https://proceedings.mlr.press/v130/yildiz21a.html},
abstract = { Learning from ranking observations arises in many domains, and siamese deep neural networks have shown excellent inference performance in this setting. However, SGD does not scale well, as an epoch grows exponentially with the ranking observation size. We show that a spectral algorithm can be combined with deep learning methods to significantly accelerate training. We combine a spectral estimate of Plackett-Luce ranking scores with a deep model via the Alternating Directions Method of Multipliers with a Kullback-Leibler proximal penalty. Compared to a state-of-the-art siamese network, our algorithms are up to 175 times faster and attain better predictions by up to 26% Top-1 Accuracy and 6% Kendall-Tau correlation over five real-life ranking datasets. }
}