Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Follow Everything: A Leader-Following and Obstacle Avoidance Framework with Goal-Aware Adaptation

Qianyi Zhang, Shijian Ma, Boyi Liu, Jianhao Jiao, Dimitrios Kanoulas

TL;DR

Robotic leader-following is challenged by arbitrary leader forms and temporary occlusion from the follower's field of view. The authors present Follow Everything, a unified framework that combines a SAM-based segmentation with a distance frame buffer, a goal-aware adaptation mechanism, and a graph-based planner to robustly track and follow leaders while avoiding obstacles. Key contributions include distance-aware segmentation memory enabling re-identification across occlusions, a multi-state adaptation that modulates $V_t^{max}$ and $D_t$ and yields dynamic goals, and a topological-graph planner that generates and optimizes time-efficient trajectories under obstacle and goal constraints. Across simulations and real-world tests with diverse leaders (human, ground robot, UAV, stop sign) in indoor and outdoor settings, the method achieves higher follow success, lower leader-loss time, fewer collisions, and tighter robot–leader distances, indicating strong potential for practical deployment.

Abstract

Robust and flexible leader-following is a critical capability for robots to integrate into human society. While existing methods struggle to generalize to leaders of arbitrary form and often fail when the leader temporarily leaves the robot's field of view, this work introduces a unified framework addressing both challenges. First, traditional detection models are replaced with a segmentation model, allowing the leader to be anything. To enhance recognition robustness, a distance frame buffer is implemented that stores leader embeddings at multiple distances, accounting for the unique characteristics of leader-following tasks. Second, a goal-aware adaptation mechanism is designed to govern robot planning states based on the leader's visibility and motion, complemented by a graph-based planner that generates candidate trajectories for each state, ensuring efficient following with obstacle avoidance. Simulations and real-world experiments with a legged robot follower and various leaders (human, ground robot, UAV, legged robot, stop sign) in both indoor and outdoor environments show competitive improvements in follow success rate, reduced visual loss duration, lower collision rate, and decreased leader-follower distance.

Follow Everything: A Leader-Following and Obstacle Avoidance Framework with Goal-Aware Adaptation

TL;DR

Robotic leader-following is challenged by arbitrary leader forms and temporary occlusion from the follower's field of view. The authors present Follow Everything, a unified framework that combines a SAM-based segmentation with a distance frame buffer, a goal-aware adaptation mechanism, and a graph-based planner to robustly track and follow leaders while avoiding obstacles. Key contributions include distance-aware segmentation memory enabling re-identification across occlusions, a multi-state adaptation that modulates and and yields dynamic goals, and a topological-graph planner that generates and optimizes time-efficient trajectories under obstacle and goal constraints. Across simulations and real-world tests with diverse leaders (human, ground robot, UAV, stop sign) in indoor and outdoor settings, the method achieves higher follow success, lower leader-loss time, fewer collisions, and tighter robot–leader distances, indicating strong potential for practical deployment.

Abstract

Robust and flexible leader-following is a critical capability for robots to integrate into human society. While existing methods struggle to generalize to leaders of arbitrary form and often fail when the leader temporarily leaves the robot's field of view, this work introduces a unified framework addressing both challenges. First, traditional detection models are replaced with a segmentation model, allowing the leader to be anything. To enhance recognition robustness, a distance frame buffer is implemented that stores leader embeddings at multiple distances, accounting for the unique characteristics of leader-following tasks. Second, a goal-aware adaptation mechanism is designed to govern robot planning states based on the leader's visibility and motion, complemented by a graph-based planner that generates candidate trajectories for each state, ensuring efficient following with obstacle avoidance. Simulations and real-world experiments with a legged robot follower and various leaders (human, ground robot, UAV, legged robot, stop sign) in both indoor and outdoor environments show competitive improvements in follow success rate, reduced visual loss duration, lower collision rate, and decreased leader-follower distance.
Paper Structure (9 sections, 13 equations, 9 figures, 1 table)

This paper contains 9 sections, 13 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Leader Segmentation with Distance Frame Buffer
  5. Graph-based Planner
  6. Goal-aware Adaptation
  7. Simulation
  8. Real-world Experiment
  9. Conclusion and Future Work

Figures (9)

  • Figure 1: Illustration of the challenges in leader-following tasks. (a) When the leader steps over a box that the follower must detour around, the leader leaves the follower’s field of view. (b) The follower is expected to follow the leader via a shorter trajectory, not by replicating the leader’s past path.
  • Figure 2: Illustration of the proposed follow everything framework. Given a leader prompt and an RGB image, the leader mask is segmented with the aid of both a memory buffer and a distance frame buffer. The segmented leader is then filtered from raw laser point clouds to produce a leader point set, which is passed to the goal-aware adaptation to dynamically adjust parameters and provide goal sets and constraints according to the follower-leader interaction. Given the costmap, goal sets, and constraints, the graph-based planner searches, optimizes, and selects an optimal trajectory for the follower to execute.
  • Figure 3: Illustration of goal-aware adaptation and graph-based planner in a demo. (a) The leader is segmented after selecting points. Chasing is activated as the leader is far away. (b) Following is triggered when the robot gets closer, maintaining a safe distance. (c) Planning is activated when the leader exits the robot's FOV, guiding the robot to the last known leader pose. (d) Following resumes once the leader is visible again. (e) Retreating is triggered when the leader steps back. (f) A new prompt "follow the person on the left side" activates Switching, followed by Chasing as the new leader is distant.
  • Figure 4: Simulation for following a mobile robot in a playground. (a) The leader can be robustly segmented even when its color closely resembles the ground. (b) Multiple candidate trajectories are generated for the robot to choose from. (c) When the leader is lost, the planning state enables the follower to actively search for the leader instead of awkwardly stopping in place. (d) Even under dim lighting conditions, the leader can still be robustly segmented.
  • Figure 5: Simulation for following a pedestrian in a factory. (a–b) The robot follows the leader when partially observed. (c) Upon receiving a prompt specifying a new leader, the chasing state is activated, and the robot passes underneath the workstation to approach the new leader. (d) When the leader steps backward, the robot retreats to maintain a safe distance. (e) When the leader turns a corner and leaves the robot's FOV, the planning state is triggered.
  • ...and 4 more figures