Autonomous Bimanual Manipulation of Deformable Objects Using Deep Reinforcement Learning Guided Adaptive Control

Abstract

Deformable object manipulation (DOM) which is a common subtask in various surgical procedures represents an inevitable challenge in robot-assisted surgery (RAS) due to complex nonlinear deformation. This paper proposes a deep reinforcement learning guided adaptive control (RLAC) modelfree framework, which combines learning-based and Jacobianbased methods. To complement each other for optimized performance, we harness the sampling of deep reinforcement learning (DRL) policy explored in simulations to solve a reasonable estimation of the initial deformation Jacobian. In early control iterations, the actions suggested by the DRL agent are adopted until the estimated real-time Jacobian approximates the actual deformation model. Subsequently, the independent Jacobianbased adaptive control (AC) with sufficient initial deformation awareness begins execution to achieve precise internal feature manipulation on deformable objects. Experimental results demonstrate that our method enables more efficient positioning and exhibits near-optimal positioning paths. RLAC with robust sim-to-real performance provides a feasible approach for the complex autonomous DOM in the real world.