Abstracted Trajectory Visualization for Explainability in Reinforcement Learning

TL;DR

The paper tackles the challenge of explaining RL agents to non-RL experts by introducing abstracted trajectory visualization. It combines a $β$-VAE–based trajectory extraction that yields latent representations $z$ and a spatio-temporal clustering step using ST-DBSCAN to identify major states, which are then decoded to form abstracted trajectories. An interactive interface with a map view and a slider view is proposed to reveal temporal dynamics; a preliminary online study indicates that abstracted trajectories support non-RL experts in inferring agent behavior as effectively as complete trajectories, with a user preference for the slider-based navigation. These findings suggest the approach can broaden participation in RL design discussions by providing concise, interpretable explanations, though usability of the map component and alignment of abstractions with human intuition require further refinement.

Abstract

Explainable AI (XAI) has demonstrated the potential to help reinforcement learning (RL) practitioners to understand how RL models work. However, XAI for users who do not have RL expertise (non-RL experts), has not been studied sufficiently. This results in a difficulty for the non-RL experts to participate in the fundamental discussion of how RL models should be designed for an incoming society where humans and AI coexist. Solving such a problem would enable RL experts to communicate with the non-RL experts in producing machine learning solutions that better fit our society. We argue that abstracted trajectories, that depicts transitions between the major states of the RL model, will be useful for non-RL experts to build a mental model of the agents. Our early results suggest that by leveraging a visualization of the abstracted trajectories, users without RL expertise are able to infer the behavior patterns of RL.

Figures (11)

• Figure 1: The process of generating abstracted trajectories from input images that a RL agent observed. The features $z$ (latent variables) are extracted from the input images $x$ with VAE (See section \ref{['subSection:Trajectory Extraction']}). The extracted features $z$ are abstracted by spatio-temporal clustering (See section \ref{['subSection:Trajectory Abstraction']}).
• Figure 2: a) The architecture of VAE, input images are fed to the decoder of VAE and the decoder is trained to compress inputs to latent variable $z$. The encoder of VAE is also trained at the same time to reconstruct the inputs from corresponding latent vectors. b) The reconstructed images projected on the latent space formed $z_1$ and $z_2$.
• Figure 3: Proposed interface for abstracted trajectory visualization: A map view is placed in the center of the interface, and a slider view is placed on the bottom. Enlarged image of a hovered over node is displayed in the inspector window on the top right.
• Figure 4: Two visualization types used in a comparative evaluation
• Figure 5: Designed analytical task in the case of abstracted trajectory: Participants are asked to find a trajectory visualized in the interface of abstracted trajectory (see region (1) in the figure) that matches an animation (see region (2) in the figure). In the case of complete trajectory, the interface in the region (1) is replaced with visualizations of complete trajectory shown in figure 4 (a).