Multi-Modal Generative Models for Learning Epistemic Active Sensing
Timo Korthals, Daniel Rudolph, Jürgen Leitner, Marc Hesse, Ulrich Rückert
Abstract
We present a novel approach of multi-modal deep generative models and apply this to coordinated heterogeneous multi-agent active sensing. A major approach to achieve this objective is to train a multi-modal variational Auto Encoder (M2VAE) that integrates the information of different sensor modalities into a joint latent representation. Furthermore, we derive an objective from the M2VAE that enables the maximization of the evidence lower bound via selection of sensor modalities. Using this approach as a direct reward signal to a multi-modal and multi-agent deep reinforcement learning setup leads intuitively to an epistemic active sensing behavior that coordinately resolves the ambiguity of observations.
BibTeX
@inproceedings{icra2019_multimodalgenera,
title = {Multi-Modal Generative Models for Learning Epistemic Active Sensing},
author = {Timo Korthals and Daniel Rudolph and Jürgen Leitner and Marc Hesse and Ulrich Rückert},
booktitle = {ICRA 2019},
year = {2019}
}