Value Iteration Networks
Aviv Tamar, YI WU, Garrett Thomas, Sergey Levine, Pieter Abbeel
Abstract
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiable approximation of the value-iteration algorithm, which can be represented as a convolutional neural network, and trained end-to-end using standard backpropagation. We evaluate VIN based policies on discrete and continuous path-planning domains, and on a natural-language based search task. We show that by learning an explicit planning computation, VIN policies generalize better to new, unseen domains.
BibTeX
@inproceedings{NIPS2016_c21002f4,
author = {Tamar, Aviv and WU, YI and Thomas, Garrett and Levine, Sergey and Abbeel, Pieter},
booktitle = {Advances in Neural Information Processing Systems},
editor = {D. Lee and M. Sugiyama and U. Luxburg and I. Guyon and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Value Iteration Networks},
url = {https://proceedings.neurips.cc/paper_files/paper/2016/file/c21002f464c5fc5bee3b98ced83963b8-Paper.pdf},
volume = {29},
year = {2016}
}