A Shopping Service Robot Framework With Visual-WEM Tracking and Intersection-Aware Following

Abstract

The shopping service robot (SSR) is designed to offer a superior shopping experience through its continuous target following and companion services. However, the SSR encounters difficulties in complex environments, including visual occlusions and spatial constraints in narrow aisles. To address these difficulties, the study proposes a shopping service robot framework that integrates innovative perception, following control, and path planning modules. Firstly, the perception module employs a multi-sensor fusion method for human target tracking, integrating both red-green-blue-depth (RGB-D) camera and wireless electromagnetic (WEM) data by using an extended Kalman filter (EKF). Secondly, the target-following control module employs an omnidirectional constrained control law, which ensures synchronized orientation alignment between the SSR and the human target. Finally, the path planning module employs topological mapping to encode intersection geometries as path nodes, thereby guiding the SSR pass through narrow shelf aisles. In real supermarkets, we evaluated the target tracking approach under shelf occlusions and the human following task within narrow aisles. Experimental results demonstrate that the visual-WEM tracking approach achieves a pose tracking accuracy of (4.56 mm, 2.98<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula>) under shelf occlusions. This study establishes the feasibility of human-robot collaboration in facilitating a hands-free shopping experience, highlighting its potential as a substitute for conventional shopping carts.