ICML 2022spotlight21 citations
Contextual Information-Directed Sampling
Botao Hao, Tor Lattimore, Chao Qin
Abstract
Information-directed sampling (IDS) has recently demonstrated its potential as a data-efficient reinforcement learning algorithm. However, it is still unclear what is the right form of information ratio to optimize when contextual information is available. We investigate the IDS design through two contextual bandit problems: contextual bandits with graph feedback and sparse linear contextual bandits. We provably demonstrate the advantage of
BibTeX
@InProceedings{pmlr-v162-hao22b,
title = {Contextual Information-Directed Sampling},
author = {Hao, Botao and Lattimore, Tor and Qin, Chao},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
pages = {8446--8464},
year = {2022},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
volume = {162},
series = {Proceedings of Machine Learning Research},
month = {17--23 Jul},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v162/hao22b/hao22b.pdf},
url = {https://proceedings.mlr.press/v162/hao22b.html},
abstract = {Information-directed sampling (IDS) has recently demonstrated its potential as a data-efficient reinforcement learning algorithm. However, it is still unclear what is the right form of information ratio to optimize when contextual information is available. We investigate the IDS design through two contextual bandit problems: contextual bandits with graph feedback and sparse linear contextual bandits. We provably demonstrate the advantage of