Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees

TL;DR

The paper tackles safe neural network control for uncertain linear time-invariant plants by enforcing closed-loop dissipativity through IQCs. It derives a dissipativity certificate for uncertain LTI systems and transforms a BMI-based controller synthesis condition into an LMIs-based formulation via a variable change, enabling a projection-based RL training loop that preserves dissipativity. The approach is instantiated with implicit/recurrent neural networks and validated on a nonlinear inverted pendulum and a flexible rod on a cart, showing competitive performance while providing formal L2 gain and stability guarantees. This contributes a practically applicable framework for training NN controllers with rigorous safety and robustness guarantees in safety-critical settings.

Abstract

In this paper, a method is presented to synthesize neural network controllers such that the feedback system of plant and controller is dissipative, certifying performance requirements such as L2 gain bounds. The class of plants considered is that of linear time-invariant (LTI) systems interconnected with an uncertainty, including nonlinearities treated as an uncertainty for convenience of analysis. The uncertainty of the plant and the nonlinearities of the neural network are both described using integral quadratic constraints (IQCs). First, a dissipativity condition is derived for uncertain LTI systems. Second, this condition is used to construct a linear matrix inequality (LMI) which can be used to synthesize neural network controllers. Finally, this convex condition is used in a projection-based training method to synthesize neural network controllers with dissipativity guarantees. Numerical examples on an inverted pendulum and a flexible rod on a cart are provided to demonstrate the effectiveness of this approach.

Figures (7)

• Figure 1: Interconnection of plant, formed of LTI system $G_p$ and uncertainty $\Delta_p$, and controller $K$.
• Figure 2: The neural network controller is modeled as the interconnection of an LTI system $G_k$ and a nonlinearity $\phi$. The nonlinearity represents the activation functions of the neural network.
• Figure 3: The original system, the interconnection of $G$ and $\Delta$, is transformed such that the transformed uncertainty $\tilde{\Delta}$ satisfies a simpler, static, IQC.
• Figure 4: Evaluation reward vs. number of training environment steps for the Dissipative Recurrent Implicit Neural Network (D-RINN, our method), Fully Connected Neural Network (FCNN), Standard RINN (S-RINN), and linear time-invariant (LTI) controllers on the inverted pendulum. The solid line represents the mean, and the shaded region represents 1 standard deviation. The reward is upper bounded by 201, and lower bounded by 0. Higher reward indicates lower control effort. The Dissipative RINN and LTI controllers both guarantee stability of the inverted pendulum.
• Figure 5: Diagram of the flexible rod on a cart.