Synergistic Task and Motion Planning With Reinforcement Learning-Based Non-Prehensile Actions

Gaoyuan Liu; Joris De Winter; Denis Steckelmacher; Roshan Kumar Hota; Ann Nowé; Bram Vanderborght

RA-L 202315 citations

Synergistic Task and Motion Planning With Reinforcement Learning-Based Non-Prehensile Actions

Gaoyuan Liu, Joris De Winter, Denis Steckelmacher, Roshan Kumar Hota, Ann Nowé, Bram Vanderborght

Abstract

Robotic manipulation in cluttered environments requires synergistic planning among prehensile and non-prehensile actions. Previous works on sampling-based Task and Motion Planning (TAMP) algorithms, e.g. PDDLStream, provide a fast and generalizable solution for multi-modal manipulation. However, they are likely to fail in cluttered scenarios where no collision-free grasping approaches can be sampled without preliminary manipulations. To extend the ability of sampling-based algorithms, we integrate a vision-based Reinforcement Learning (RL) non-prehensile procedure, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pusher . The pushing actions generated by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pusher can eliminate interlocked situations and make the grasping problem solvable. Also, the sampling-based algorithm evaluates the pushing actions by providing rewards in the training process, thus the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pusher can learn to avoid situations leading to irreversible failures. The proposed hybrid planning method is validated on a cluttered bin-picking problem and implemented in both simulation and real world. Results show that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pusher can effectively improve the success ratio of the previous sampling-based algorithm, while the sampling-based algorithm can help the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pusher learn pushing skills.

BibTeX

@inproceedings{ral2023_synergistictaska,
  title = {Synergistic Task and Motion Planning With Reinforcement Learning-Based Non-Prehensile Actions},
  author = {Gaoyuan Liu and Joris De Winter and Denis Steckelmacher and Roshan Kumar Hota and Ann Nowé and Bram Vanderborght},
  booktitle = {RA-L 2023},
  year = {2023}
}