Realistic Counterfactual Explanations for Machine Learning-Controlled Mobile Robots using 2D LiDAR

TL;DR

This work addresses the interpretability gap in ML-controlled mobile robotics by generating realistic counterfactual explanations for 2D LiDAR inputs. It introduces a model-agnostic framework that parameterizes LiDAR space with geometric shapes and uses a genetic algorithm to place virtual obstacles, producing CFEs that induce predefined model outputs via raycasting-based LiDAR simulations. The approach includes two data-combination strategies, a specialized fitness function with hinge and proximity losses, and a comprehensive TurtleBot3 case study using DRL (SAC) trained with domain randomization. Compared to DiCE, the method yields more plausible, human-understandable explanations, validated through both simulated and real-world experiments, including cases with no obstacles, added obstacles to trigger crashes, and explanations of left-turn bias. The results demonstrate improved interpretability and practical utility for debugging and refining ML-based autonomous control, with future work extending to 3D LiDAR and broader robotic applications.

Abstract

This paper presents a novel method for generating realistic counterfactual explanations (CFEs) in machine learning (ML)-based control for mobile robots using 2D LiDAR. ML models, especially artificial neural networks (ANNs), can provide advanced decision-making and control capabilities by learning from data. However, they often function as black boxes, making it challenging to interpret them. This is especially a problem in safety-critical control applications. To generate realistic CFEs, we parameterize the LiDAR space with simple shapes such as circles and rectangles, whose parameters are chosen by a genetic algorithm, and the configurations are transformed into LiDAR data by raycasting. Our model-agnostic approach generates CFEs in the form of synthetic LiDAR data that resembles a base LiDAR state but is modified to produce a pre-defined ML model control output based on a query from the user. We demonstrate our method on a mobile robot, the TurtleBot3, controlled using deep reinforcement learning (DRL) in real-world and simulated scenarios. Our method generates logical and realistic CFEs, which helps to interpret the DRL agent's decision making. This paper contributes towards advancing explainable AI in mobile robotics, and our method could be a tool for understanding, debugging, and improving ML-based autonomous control.

Figures (11)

• Figure 1: Example illustrating the two combination methods. The top left shows the base LiDAR state with four walls around the sensor, while the top right shows the generated virtual obstacles. The lower left shows the result of combining these data with min_distance, which keeps the nearest obstacles at each reading, and the lower right shows gen_priority, which prioritizes the virtual obstacles. The dot at the center marks the origin of the LiDAR sensor.
• Figure 2: Example of a Gazebo scene and the corresponding LiDAR plot. The goal in front of the TurtleBot3 is shown as a red square in Gazebo and an orange circle in the LiDAR plot. The TurtleBot3 is represented by the black dot in the middle of the plot, the blue square to the left of the TurtleBot3 is shown to the left of the TurtleBot3 in the plot, and vice versa with the orange cylinder to the right. The grey sphere is not visible in \ref{['subfig:gazebo_lidar_comparison_lidar']} since it is behind the blue cube and not visible from the TurtleBot3's point of view.
• Figure 3: Photos of the real-world setup for Case 2, before and after actualizing the cfe, discussed in \ref{['subsubsec:case_2']}.
• Figure 4: Comparison between dice and our algorithm on a simple case where the goal is right in front, with walls on either side.
• Figure 5: Four cfes generated by the algorithm from the base state in Case 1, with the desired action being to move backwards, and not turn much, which we define by $[a_{\text{linear}} \in [-1.0, 0.0]$ and $a_{\text{angular}} \in [-0.2, 0.2]$.