2024
Computing the Bias of Constant-step Stochastic Approximation with Markovian Noise
NeurIPS 2024poster
We study stochastic approximation algorithms with Markovian noise and constant step-size $\alpha$. We develop a method based on infinitesimal generator comparisons to study the bias of the algorithm, which is the expected difference between $\theta_n$ ---the value at iteration $n$--- and $\theta^*$…