eXplainable AI for data driven control: an inverse optimal control approach

TL;DR

The paper tackles the challenge of explaining data-driven, dynamic controllers by introducing eXplainable Data-Driven Control (XDDC), a local explanation framework that linearizes around a trajectory and uses inverse LQR to recover cost weights $Q$ and $R$ as the explanatory content. By adopting Linear Time-Invariant (LTI) models as the interpretable class and employing interpretability functions such as inverse LQR, the authors connect traditional control insights with XAI concepts, enabling explanations that reflect time-dependent trade-offs in the closed-loop. Through two numerical studies—a second-order LTI system and an inverted pendulum—the approach reveals how the inferred cost terms and their (lack of) diagonal structure illuminate why a black-box/controller-tuned policy behaves as observed, including cases where non-diagonal coupling is essential for faithful explanations. The work argues for a principled, reduced-complexity representation of explanations in AI-driven control and points to future work in extending to nonlinear dynamics, improving scalability, and applying the framework to real-time, high-dimensional systems.

Abstract

Understanding the behavior of black-box data-driven controllers is a key challenge in modern control design. In this work, we propose an eXplainable AI (XAI) methodology based on Inverse Optimal Control (IOC) to obtain local explanations for the behavior of a controller operating around a given region. Specifically, we extract the weights assigned to tracking errors and control effort in the implicit cost function that a black-box controller is optimizing, offering a more transparent and interpretable representation of the controller's underlying objectives. This approach presents connections with well-established XAI techniques, such as Local Interpretable Model-agnostic Explanations (LIME) since it is still based on a local approximation of the control policy. However, rather being limited to a standard sensitivity analysis, the explanation provided by our method relies on the solution of an inverse Linear Quadratic (LQ) problem, offering a structured and more control-relevant perspective. Numerical examples demonstrate that the inferred cost function consistently provides a deeper understanding of the controller's decision-making process, shedding light on otherwise counterintuitive or unexpected phenomena.

Figures (3)

• Figure 1: Schematic formal representation of an explanation method.
• Figure 2: LQR trajectories using MPC weights $Q_{MPC}$, $R_{MPC}$ and using IOC weights $\hat{Q}$, $\hat{R}$.
• Figure 3: MPC and LQR trajectories for the pendulum near the equilibrium point $\bar{\theta}_1 = 0$ rad.