SHOPPER: Practical Insights on Grasp Strategies for Mobile Manipulation in the Wild

Abstract

Mobile manipulation systems have advanced significantly in recent years. However, substantial gaps remain that prevent state-of-the-art platforms from achieving widespread real-world deployment, particularly in reliably grasping items in unstructured environments. To help bridge this gap, we develop SHOPPER, a mobile manipulation robot platform designed to push the boundaries of reliable and generalizable grasp strategies. We develop these grasp strategies and deploy them in a real-world grocery store--an exceptionally challenging setting chosen for its vast diversity of manipulable items, fixtures, and layouts. In this work, we present our detailed approach to designing general grasp strategies towards picking any item in a real grocery store. Additionally, we provide an in-depth analysis of our latest real-world field test, discussing key findings related to fundamental failure modes over hundreds of distinct pick attempts. Through our detailed analysis, we aim to offer valuable practical insights and identify key grasping challenges, which can guide the robotics community towards pressing open problems in the field. Lastly, we provide a dataset of 1200+ grasp attempts in unseen grocery stores.