Differentiable Optimization-Based Modular Planning Framework for Pick-And-Place with Regrasp

Abstract

Robotic manipulation commonly involves pick-and-place tasks in which regrasp may be necessary for low-dexterity manipulators. Many existing approaches rely on sampling, which becomes inefficient when repeated regrasp is required in high-dimensional configuration spaces. We propose a modular planning framework that comprises differentiable optimization-based modules: grasp generation, stable pose prediction, inverse kinematics solving, and path planning. The modular design yields a systematic pipeline, enabling direct pick-and-place, static or non-static release, and repeated regrasp by solving each module as needed. Each module leverages differentiable geometric features to efficiently solve its corresponding optimization problem. Our framework explicitly accounts for grasp constraints across both task scenes and predicts stable poses for regrasp planning via optimization rather than expensive physics simulations, thereby improving the feasibility and efficiency of planning. We validated the framework in pick-and-place simulations and real-world experiments.