An Evaluation of Action Segmentation Algorithms on Bimanual Manipulation Datasets

Abstract

Humans naturally execute many everyday manipulation actions with both arms simultaneously. Similarly, endowing robots with bimanual manipulation task models is key to efficiently perform complex manipulation tasks. To do so, a promising approach is to learn a library of task models from human demonstrations. However, this requires human motions to be meaningfully segmented. In this paper, we propose to segment the motion of each hand individually to account for different bimanual coordination patterns and provide a thorough evaluation of state-of-the-art segmentation algorithms on bimanual manipulation datasets. In particular, we compare segmentation algorithms at trajectory and semantic level with hierarchical algorithms. Moreover, our evaluation extensively studies the performances of various segmentation algorithms over a novel extension of the KIT Bimanual Manipulation Dataset featuring ~ 176 minutes of human motion recordings in household scenarios.