HPPS: A Hierarchical Progressive Perception System for Luggage Trolley Detection and Localization at Airports

TL;DR

HPPS addresses luggage trolley detection and localization under partial occlusion using monocular RGB input by separating position and orientation estimation. It combines a detection-keypoint pipeline with a model-based 3D location from 2D keypoints, and Gaussian-regressed orientation, all fused through a Modified Moving Average Filter and a Multi-Risk-RRT planner for real-time robot control. A 13740-image luggage-trolley dataset supports training, and real-world robot trials demonstrate robustness in complex airport-like scenes, outperforming occlusion-sensitive baselines. The approach reduces sensing requirements while delivering reliable pose estimates and end-to-end applicability to autonomous trolley collection in airports, with potential extensions to multi-robot perception and planning.

Abstract

The robotic autonomous luggage trolley collection system employs robots to gather and transport scattered luggage trolleys at airports. However, existing methods for detecting and locating these luggage trolleys often fail when they are not fully visible. To address this, we introduce the Hierarchical Progressive Perception System (HPPS), which enhances the detection and localization of luggage trolleys under partial occlusion. The HPPS processes the luggage trolley's position and orientation separately, which requires only RGB images for labeling and training, eliminating the need for 3D coordinates and alignment. The HPPS can accurately determine the position of the luggage trolley with just one well-detected keypoint and estimate the luggage trolley's orientation when it is partially occluded. Once the luggage trolley's initial pose is detected, HPPS updates this information continuously to refine its accuracy until the robot begins grasping. The experiments on detection and localization demonstrate that HPPS is more reliable under partial occlusion compared to existing methods. Its effectiveness and robustness have also been confirmed through practical tests in actual luggage trolley collection tasks. A website about this work is available at HPPS.

Figures (13)

• Figure 1: Schematic diagram of HPPS for luggage trolley detection and localization under partial occlusion at airports.
• Figure 2: A diagram of our proposed luggage trolley detection and localization system. The process starts with capturing RGB images, followed by a detection phase. Next, the system processes these detection information to determine the luggage trolley's pose, which is then filtered and progressively updated. Finally, the planning module computes and sends velocity commands to the robot in real time.
• Figure 3: An example of estimating the luggage trolley's position through keypoint observation. The process utilizes any visible keypoint for the luggage trolley's position estimation. The parameters involved and the estimation method are detailed in Eq. \ref{['estimation']}.
• Figure 4: An example of estimating the luggage trolley's orientation from a top-down perspective. The yellow curve represents the Gaussian regression probability distribution of the orientation, while the red dotted line with an arrow indicates the orientation with the highest probability.
• Figure 5: A workflow comparison of our method and Xiao's method for estimating the pose of the luggage trolley from image inputs.