Situational Graphs for Robotic First Responders: an application to dismantling drug labs

TL;DR

This paper tackles the safety-critical challenge of dismantling illicit drug laboratories by enabling safer initial investigations with mobile robots. It introduces the Behavior-Oriented Situational Graph, a robot-centric environmental representation that merges perception-driven data with a situational affordance schema to map actionable behaviors to encountered situations, thereby facilitating real-time planning and operator understanding. The core contributions include (i) formalizing the Situational Graph structure, (ii) a real-time data-collection and graph-update pipeline, (iii) a planning framework that integrates job selection and path planning over the graph, and (iv) an operator interface and immersive teleoperation workflow that supports seamless adjustment of autonomy levels. Through a formative study with police stakeholders in a mock drug-lab scenario, the approach demonstrates improved situational awareness, trustworthy autonomous exploration, and practical pathways for human–robot collaboration in high-stakes environments.

Abstract

In this work, we support experts in the safety domain with safer dismantling of drug labs, by deploying robots for the initial inspection. Being able to act on the discovered environment is key to enabling this (semi-)autonomous inspection, e.g. to open doors or take a closer at suspicious items. Our approach addresses this with a novel environmental representation, the Behavior-Oriented Situational Graph, where we extend on the classical situational graph by merging a perception-driven backbone with prior actionable knowledge via a situational affordance schema. Linking situations to robot behaviors facilitates both autonomous mission planning and situational understanding of the operator. Planning over the graph is easier and faster, since it directly incorporates actionable information, which is critical for online mission systems. Moreover, the representation allows the human operator to seamlessly transition between different levels of autonomy of the robot, from remote control to behavior execution to full autonomous exploration. We test the effectiveness of our approach in a real-world drug lab scenario at a Dutch police training facility using a mobile Spot robot and use the results to iterate on the system design.

Figures (8)

• Figure 1: Top: photo from a real drugslab (source:politie.nl). Bottom: Robot in operation during experiment in mock-up drugslab.
• Figure 2: The envisioned usage of robots in a drug lab exploration scenario, including improvements identified in this formative study. Multiple robots collaborate to explore the environment, intervene on obstacles such as closed doors, and execute mission-relevant jobs such as inspection.
• Figure 3: An example of how the Behavior-Oriented Situational Graph $\mathcal{G}$ encodes potential behaviors $b_k$ in the environment. Conventional edges $e$ between two nodes are visualized in red. Behaviors that take world objects as parameters are visualised by the blue arrows, connecting a node $v$ with a world object $o$. The robot can e.g. goTo and explore a frontier ($o_1$) at $v_2$ , open a door ($o_3$) at node $v_4$ or request the operator for assistance on a chemical container ($o_2$) from node $v_3$.
• Figure 4: A simplified overview of the centralized architecture of the system. In the agent app running on the backpack PC a perceptions server publishes perception events from sensor data, which the sgraph-recording-server uses to update the Situational Graph. On a central app outside of the drug lab, the operator-server hosts the graphical user interface for the operator and allows the operator to select the level of autonomy. Based on this level, the planning-server and execution-server allocate and execute jobs respectively. Finally, when in teleop mode, the spot-teleop server allows direct communication with the spot API to improve teleop latency.
• Figure 5: The operator interface. The 3D scene in the center shows the Situational Graph. Red nodes are waypoints visited by the robot, green nodes are unexplored frontiers and blue nodes are detected objects. Detections are represented by a unit vector from the position where the detection was done towards the detected object. The left side shows a timeline of events that occurred during the mission. The bottom elements house general options for the system, such as toggling layers on and off. The right side shows robot-specific elements such as the live stream (with detections), commands, and basic teleoperation controls.