Learning Needle Pick-and-Place Without Expert Demonstrations
Rokas Bendikas, Valerio Modugno, Dimitrios Kanoulas, Francisco Vasconcelos, Danail Stoyanov
Abstract
We introduce a novel approach for learning a complex multi-stage needle pick-and-place manipulation task for surgical applications using Reinforcement Learning without expert demonstrations or explicit curriculum. The proposed method is based on a recursive decomposition of the original task into a sequence of sub-tasks with increasing complexity and utilizes an actor-critic algorithm with deterministic policy output. In this work, exploratory bottlenecks have been used by a human expert as convenient boundary points for partitioning complex tasks into simpler subunits. Our method has successfully learnt a policy for the needle pick-and-place task, whereas the state-of-the-art TD3+HER method is unable to achieve success without the help of expert demonstrations. Comparison results show that our method achieves the highest performance with a 91% average success rate.
BibTeX
@inproceedings{ral2023_learningneedlepi,
title = {Learning Needle Pick-and-Place Without Expert Demonstrations},
author = {Rokas Bendikas and Valerio Modugno and Dimitrios Kanoulas and Francisco Vasconcelos and Danail Stoyanov},
booktitle = {RA-L 2023},
year = {2023}
}