Improved Worst-Case Regret Bounds for Randomized Least-Squares Value Iteration

Priyank Agrawal; Jinglin Chen; Nan Jiang

AAAI 2021technical24 citations

Improved Worst-Case Regret Bounds for Randomized Least-Squares Value Iteration

Priyank Agrawal, Jinglin Chen, Nan Jiang

Abstract

This paper studies regret minimization with randomized value functions in reinforcement learning. In tabular finite-horizon Markov Decision Processes, we introduce a clipping variant of one classical Thompson Sampling (TS)-like algorithm, randomized least-squares value iteration (RLSVI). Our $tilde{mathrm{O}}(H^2Ssqrt{AT})$ high-probability worst-case regret bound improves the previous sharpest worst-case regret bounds for RLSVI and matches the existing state-of-the-art worst-case TS-based regret bounds.

BibTeX

@inproceedings{aaai2021_improvedworstcas,
  title = {Improved Worst-Case Regret Bounds for Randomized Least-Squares Value Iteration},
  author = {Priyank Agrawal and Jinglin Chen and Nan Jiang},
  booktitle = {AAAI 2021},
  year = {2021}
}