Learning Multimodal Representations for Sample-efficient Recognition of Human Actions
Miguel Vasco, Francisco S. Melo, David Martins de Matos, Ana Paiva, Tetsunari Inamura
Abstract
Humans interact in rich and diverse ways with the environment. However, the representation of such behavior by artificial agents is often limited. In this work we present motion concepts, a novel multimodal representation of human actions in a household environment. A motion concept encompasses a probabilistic description of the kinematics of the action along with its contextual background, namely the location and the objects held during the performance. We introduce a novel algorithm which learns and recognizes motion concepts from action demonstrations, named Online Motion Concept Learning (OMCL). The algorithm is evaluated on a virtual-reality household environment with the presence of a human avatar. OMCL outperforms standard motion recognition algorithms on an one-shot recognition task, attesting to its potential for sample-efficient recognition of human actions.
BibTeX
@inproceedings{iros2019_learningmultimod,
title = {Learning Multimodal Representations for Sample-efficient Recognition of Human Actions},
author = {Miguel Vasco and Francisco S. Melo and David Martins de Matos and Ana Paiva and Tetsunari Inamura},
booktitle = {IROS 2019},
year = {2019}
}