Online Object-Level Semantic Mapping for Quadrupeds in Real-World Environments

TL;DR

The paper addresses real-time semantic mapping for quadruped robots operating in indoor environments, emphasizing object-level representations to support goal-directed navigation. It integrates a lightweight SLAM-based occupancy map with a D435-YOLO perception loop and a memory-based object association layer to create a persistent, queryable object layer in the map frame. The approach avoids heavy 3D fusion and maintains robustness by per-frame de-duplication and long-term memory, ensuring object instances persist across viewpoint changes. Experimental validation on a Boston Dynamics Spot demonstrates real-time operation and stable object instances despite sensor bandwidth and rate variations. Limitations include depth misalignment between depth sensors and LiDAR and reliance on detector centers, with future work targeting autonomous integration with planners.

Abstract

We present an online semantic object mapping system for a quadruped robot operating in real indoor environments, turning sensor detections into named objects in a global map. During a run, the mapper integrates range geometry with camera detections, merges co-located detections within a frame, and associates repeated detections into persistent object instances across frames. Objects remain in the map when they are out of view, and repeated sightings update the same instance rather than creating duplicates. The output is a compact object layer that can be queried (class, pose, and confidence), is integrated with the occupancy map and readable by a planner. In on-robot tests, the layer remained stable across viewpoint changes.

Figures (6)

• Figure 1: Spot by Boston Dynamics with onboard payload: 1. Intel RealSense T265 Camera, 2. Intel RealSense D435 Camera, 3. 2D LiDAR, 4. Intel NUC 11 i7-1165G7, and 5. A lithium-ion battery to power the NUC.
• Figure 2: Overview of the proposed methodology. A 2D occupancy grid map is built using 2D LiDAR scans and odometry obtained from the T265 camera. Object detections are obtained from RGB images, and depth is used to estimate 3D positions. The semantic layer places detected objects on the map, merges nearby ones, and keeps them over time.
• Figure 3: Core settings for association and memory.
• Figure 4: Association pipeline for confirmed objects.
• Figure 5: Setup and output. The semantic layer tracked only person and chair.