Semantic Belief Behavior Graph: Enabling Autonomous Robot Inspection in Unknown Environments

TL;DR

SB2G tackles autonomous inspection in unknown environments by maintaining a joint geometric and semantic belief and organizing control policies as a graph of behaviors. It introduces an active semantic search that reduces uncertainty before executing semantic inspections, enabling reliable transitions to tasks like inspection of fire extinguishers, doors, and stairs. The approach combines multiple object-specific search behaviors with geometric coverage and belief-based transitions, and is validated in Gazebo simulations and real-world field tests with a legged robot, achieving efficient inspection performance comparable to human operators under perceptual uncertainty. This work advances semantic-aware autonomous inspection and offers a scalable framework for unknown urban environments.

Abstract

This paper addresses the problem of autonomous robotic inspection in complex and unknown environments. This capability is crucial for efficient and precise inspections in various real-world scenarios, even when faced with perceptual uncertainty and lack of prior knowledge of the environment. Existing methods for real-world autonomous inspections typically rely on predefined targets and waypoints and often fail to adapt to dynamic or unknown settings. In this work, we introduce the Semantic Belief Behavior Graph (SB2G) framework as a novel approach to semantic-aware autonomous robot inspection. SB2G generates a control policy for the robot, featuring behavior nodes that encapsulate various semantic-based policies designed for inspecting different classes of objects. We design an active semantic search behavior to guide the robot in locating objects for inspection while reducing semantic information uncertainty. The edges in the SB2G encode transitions between these behaviors. We validate our approach through simulation and real-world urban inspections using a legged robotic platform. Our results show that SB2G enables a more efficient inspection policy, exhibiting performance comparable to human-operated inspections.

Figures (6)

• Figure 1: Autonomous robot inspection in urban environments. This figure showcases various key semantic-aware behaviors performed by our robot to enable autonomous inspections.
• Figure 2: The system architecture of SB2G. SB2G gathers belief state $b$ from various perception modules. SB2G selects a behavior based on $b$ and uses belief prediction $\tau$ to compute a robot control policy $\pi$. The policy controls robot locomotion, high-resolution data capture, and lighting modules.
• Figure 3: Comparison of robot paths for object inspections using SB2G and baseline methods in a simulation. Our method successfully locates and inspects all objects while following a shorter trajectory.
• Figure 4: The number of inspected objects (left) and the sum of the closest distance to the all 10 objects over time (right). The closest distance to each object is initially set to $5$ m.
• Figure 5: Experimental results of real-world autonomous inspections using SB2G. The left figure compares the SB2G's robot paths with paths manually operated by a human during the search and inspection of fire extinguishers. The middle figures show our legged robot performing geometric coverage, semantic search, fire extinguisher inspection, and stair climbing. The right figures provide the camera view of the robot, demonstrating its capability to inspect the gauge of the fire extinguisher.