Non-Stationary Bandits with Periodic Behavior: Harnessing Ramanujan Periodicity Transforms to Conquer Time-Varying Challenges

Parth Thaker; Vineet Sunil Gattani; Vignesh Tirukkonda; Pouria Saidi; Gautam Dasarathy

ICASSP 2024accepted0 citations

Non-Stationary Bandits with Periodic Behavior: Harnessing Ramanujan Periodicity Transforms to Conquer Time-Varying Challenges

Parth Thaker, Vineet Sunil Gattani, Vignesh Tirukkonda, Pouria Saidi, Gautam Dasarathy

Abstract

In traditional multi-armed bandits (MAB), a standard assumption is that the mean rewards are constant across each arm, a simplification that can be restrictive in nature. In many real-world settings, the rewards exhibit a periodic pattern on which traditional MAB algorithms would fail. This paper addresses the problem of regret minimization when the mean rewards change periodically. To this end, we propose an approach that utilizes the Ramanujan periodicity transform to estimate the support of the periods efficiently and, furthermore, use this information to minimize regret.

BibTeX

@inproceedings{icassp2024_nonstationaryban,
  title = {Non-Stationary Bandits with Periodic Behavior: Harnessing Ramanujan Periodicity Transforms to Conquer Time-Varying Challenges},
  author = {Parth Thaker and Vineet Sunil Gattani and Vignesh Tirukkonda and Pouria Saidi and Gautam Dasarathy},
  booktitle = {ICASSP 2024},
  year = {2024}
}