Online Learning in Contextual Bandits using Gated Linear Networks
Eren Sezener, Marcus Hutter, David Budden, Jianan Wang, Joel Veness
Abstract
We introduce a new and completely online contextual bandit algorithm called Gated Linear Contextual Bandits (GLCB). This algorithm is based on Gated Linear Networks (GLNs), a recently introduced deep learning architecture with properties well-suited to the online setting. Leveraging data-dependent gating properties of the GLN we are able to estimate prediction uncertainty with effectively zero algorithmic overhead. We empirically evaluate GLCB compared to 9 state-of-the-art algorithms that leverage deep neural networks, on a standard benchmark suite of discrete and continuous contextual bandit problems. GLCB obtains mean first-place despite being the only online method, and we further support these results with a theoretical study of its convergence properties.
BibTeX
@inproceedings{NEURIPS2020_e287f0b2,
author = {Sezener, Eren and Hutter, Marcus and Budden, David and Wang, Jianan and Veness, Joel},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {19467--19477},
publisher = {Curran Associates, Inc.},
title = {Online Learning in Contextual Bandits using Gated Linear Networks},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/e287f0b2e730059c55d97fa92649f4f2-Paper.pdf},
volume = {33},
year = {2020}
}