Follow Everything: Goal-Aware Adaptation and Graph-Based Planner for Arbitrary Leader Following

Abstract

Enabling robots to robustly follow leaders supports tasks such as carrying supplies or guiding customers. While existing methods often fail to generalize to arbitrary leaders, and struggle when the leader temporarily leaves the robot’s field of view, this work presents a unified framework to address both challenges. First, a segmentation model replaces traditional category-specific detection models, allowing the leader to be of any shape or type. To improve robustness, a distance frame buffer is designed to store high-confidence leader embeddings across distance intervals, accounting for the unique characteristics of leader-following tasks. Second, a goal-aware adaptation mechanism is designed to govern robot planning states based on the leader's visibility and motion, complemented by a graph-based planner that generates candidate trajectories for each state, ensuring efficient following with obstacle avoidance. Simulations and real-world experiments with a legged robot follower and diverse leaders in indoor and outdoor settings demonstrate the highest follow success rate of 96.9%, the lowest visual loss of 10.7%, the lowest collision rate of 1.8%, and the shortest leader-follower distance of 2.0 m. Visit follow-everything.github.io for more details.