Vagul Mahadevan

Convergence of Two-Timescale Stochastic Approximation with Markovian Samples and Applications in Reinforcement Learning

Stochastic approximations (SA)--algorithms which derive their power through the use of random, incremental updates--are at the heart of reinforcement learning (RL). Expanding the theory of SA has established rigorous results concerning the most important algorithms in RL, including stochastic gradie…