Transfer Learning for Control Systems via Neural Simulation Relations

TL;DR

This work tackles transferring control policies between related dynamical systems with formal guarantees by learning neural simulation relations. By framing the transfer via an $\epsilon$-approximate simulation relation and an interface function, the authors train neural nets $V$ and $\mathcal{K}$ to certify and realize behavior transfer from a black-box source system to a target system. The key contributions are the data-driven construction of a neural simulation relation, the associated interface, and formal validity conditions that imply an $\epsilon$-bounded output discrepancy without post-hoc verification. The framework is demonstrated on a Vehicle model and a Double Inverted Pendulum, achieving $\epsilon$-level closeness and safe transfer under practical Lipschitz assumptions, highlighting potential for safe controller reuse in safety-critical, partially specified settings. This approach enables robust transfer in settings with digital twins or black-box dynamics, reducing the need for explicit model-based guarantees while maintaining rigorous bounds on performance.

Abstract

Transfer learning is an umbrella term for machine learning approaches that leverage knowledge gained from solving one problem (the source domain) to improve speed, efficiency, and data requirements in solving a different but related problem (the target domain). The performance of the transferred model in the target domain is typically measured via some notion of loss function in the target domain. This paper focuses on effectively transferring control logic from a source control system to a target control system while providing approximately similar behavioral guarantees in both domains. However, in the absence of a complete characterization of behavioral specifications, this problem cannot be captured in terms of loss functions. To overcome this challenge, we use (approximate) simulation relations to characterize observational equivalence between the behaviors of two systems. Simulation relations ensure that the outputs of both systems, equipped with their corresponding controllers, remain close to each other over time, and their closeness can be quantified {\it a priori}. By parameterizing simulation relations with neural networks, we introduce the notion of \emph{neural simulation relations}, which provides a data-driven approach to transfer any synthesized controller, regardless of the specification of interest, along with its proof of correctness. Compared with prior approaches, our method eliminates the need for a closed-loop mathematical model and specific requirements for both the source and target systems. We also introduce validity conditions that, when satisfied, guarantee the closeness of the outputs of two systems equipped with their corresponding controllers, thus eliminating the need for post-facto verification. We demonstrate the effectiveness of our approach through case studies involving a vehicle and a double inverted pendulum.

Figures (4)

• Figure 1: The proposed behavior transfer framework for controller synthesis. The existence of a relation and a interface function between source and target systems imply closeness of their behavior.
• Figure 2: Figure \ref{['fig_abs_error']} depicts the error between the outputs, and Figure \ref{['fig_relative']} depicts the trajectories for both systems. Red area depicts the unsafe set.
• Figure 3: Source and target systems.
• Figure 4: Multiple trajectories of target and source systems are depicted in Figure \ref{['fig_relative_dip']}, and their corresponding output error is depicted in Figure \ref{['fig_abs_error_dip']}.

Theorems & Definitions (9)

• Definition 1
• Definition 3
• Definition 5: Approximate Simulation Relation
• Proposition 6: Approximate Simulation Relation Imply Transferabilitytabuada2009verification
• Definition 7
• Definition 8
• Theorem 10
• proof
• Remark 11