Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Strategy Synthesis for Switched Stochastic Systems with Distributional Uncertainty

Ibon Gracia, Dimitris Boskos, Morteza Lahijanian, Luca Laurenti, Manuel Mazo

TL;DR

The paper develops a formal framework for distributionally robust control of discrete-time switched stochastic systems with additive noise whose distribution lies in a Wasserstein ambiguity set around a nominal model. It builds a finite robust MDP abstraction by discretizing the state and lifting distributional uncertainty via optimal transport, then solves robust reach-avoid problems using a dual LP-enhanced robust dynamic programming approach, with proofs of convergence and correctness for both finite and infinite horizons. The abstraction is refined back to the original system, guaranteeing probabilistic bounds on satisfaction under all distributions in the ambiguity set. Experiments on linear and nonlinear switched systems, including complex LTL_f specifications, demonstrate accurate guarantees and significant computational gains over standard LP methods and IMDP-based approaches, underscoring practical viability for safety-critical settings.

Abstract

We introduce a framework for the control of discrete-time switched stochastic systems with uncertain distributions. In particular, we consider stochastic dynamics with additive noise whose distribution lies in an ambiguity set of distributions that are $\varepsilon-$close, in the Wasserstein distance sense, to a nominal one. We propose algorithms for the efficient synthesis of distributionally robust control strategies that maximize the satisfaction probability of reach-avoid specifications with either a given or an arbitrary (not specified) time horizon, i.e., unbounded-time reachability. The framework consists of two main steps: finite abstraction and control synthesis. First, we construct a finite abstraction of the switched stochastic system as a \emph{robust Markov decision process} (robust MDP) that encompasses both the stochasticity of the system and the uncertainty in the noise distribution. Then, we synthesize a strategy that is robust to the distributional uncertainty on the resulting robust MDP. We employ techniques from optimal transport and stochastic programming to reduce the strategy synthesis problem to a set of linear programs, and propose a tailored and efficient algorithm to solve them. The resulting strategies are correctly refined into switching strategies for the original stochastic system. We illustrate the efficacy of our framework on various case studies comprising both linear and non-linear switched stochastic systems.

Efficient Strategy Synthesis for Switched Stochastic Systems with Distributional Uncertainty

TL;DR

The paper develops a formal framework for distributionally robust control of discrete-time switched stochastic systems with additive noise whose distribution lies in a Wasserstein ambiguity set around a nominal model. It builds a finite robust MDP abstraction by discretizing the state and lifting distributional uncertainty via optimal transport, then solves robust reach-avoid problems using a dual LP-enhanced robust dynamic programming approach, with proofs of convergence and correctness for both finite and infinite horizons. The abstraction is refined back to the original system, guaranteeing probabilistic bounds on satisfaction under all distributions in the ambiguity set. Experiments on linear and nonlinear switched systems, including complex LTL_f specifications, demonstrate accurate guarantees and significant computational gains over standard LP methods and IMDP-based approaches, underscoring practical viability for safety-critical settings.

Abstract

We introduce a framework for the control of discrete-time switched stochastic systems with uncertain distributions. In particular, we consider stochastic dynamics with additive noise whose distribution lies in an ambiguity set of distributions that are close, in the Wasserstein distance sense, to a nominal one. We propose algorithms for the efficient synthesis of distributionally robust control strategies that maximize the satisfaction probability of reach-avoid specifications with either a given or an arbitrary (not specified) time horizon, i.e., unbounded-time reachability. The framework consists of two main steps: finite abstraction and control synthesis. First, we construct a finite abstraction of the switched stochastic system as a \emph{robust Markov decision process} (robust MDP) that encompasses both the stochasticity of the system and the uncertainty in the noise distribution. Then, we synthesize a strategy that is robust to the distributional uncertainty on the resulting robust MDP. We employ techniques from optimal transport and stochastic programming to reduce the strategy synthesis problem to a set of linear programs, and propose a tailored and efficient algorithm to solve them. The resulting strategies are correctly refined into switching strategies for the original stochastic system. We illustrate the efficacy of our framework on various case studies comprising both linear and non-linear switched stochastic systems.
Paper Structure (29 sections, 12 theorems, 80 equations, 7 figures, 1 table)

This paper contains 29 sections, 12 theorems, 80 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Previous Work
  3. Related Work
  4. Basic Notation
  5. Problem Formulation
  6. Overview of the Approach
  7. Preliminaries
  8. Robust Markov Decision Processes
  9. Robust MDP Abstraction
  10. States and Actions
  11. Transition Probability Distributions
  12. Accounting for the Discretization Error
  13. Accounting for the Distributional Uncertainty
  14. Robust Strategy Synthesis
  15. Robust Dynamic Programming
  16. ...and 14 more sections

Key Result

Proposition 5.2

Consider the robust MDP abstraction $\mathcal{M} = (Q,A,\Gamma)$ of System eq:system. Let $q\in Q_{\rm{safe}}$, $a\in A$, $x\in q$, and $p_{v}\in\mathcal{P}_v$, and define $\gamma_{x,a}\in\mathcal{D}(Q)$ as for all $q'\in Q$. Then $\gamma_{x,a}\in\Gamma_{q,a}$.

Figures (7)

  • Figure 1: Results of Experiments $\#1-\#6$ in Table \ref{['tab:results']}. In \ref{['fig:P_reach_lower_unicycle_R_6']}, \ref{['fig:P_reach_lower_unicycle_R_5']}, and \ref{['fig:P_reach_lower_unicycle_R_7']}, lower bound in the probability of reachability corresponding to Experiments $\#1$ (and $\#2$), $\#3$ (and $\#4$) and $\#5$ (and $\#6$), respectively. The plotted trajectories correspond to Monte Carlo simulations of System \ref{['eq:unicycle_model']} taking samples from a distribution $\widetilde{p}_v\in\mathcal{P}_v$. In \ref{['fig:P_reach_upper_unicycle2D']}, upper bound in the probability of reachability corresponding to Experiments $\#1-\#6$. In \ref{['fig:strategy_unicycle_R_6']}, optimal strategy of Experiments $\#1$ (and $\#2$).
  • Figure 2: Results of experiments $\#1$ and $\#7$ in Table \ref{['tab:results']}. Lower bound in the probability of reachability. The trajectories in both figures correspond to Monte Carlo simulations taking samples from a distribution $\widetilde{p}_v\in\mathcal{P}_v$. The ones that satisfy the specification are presented in blue, while the ones that do not are presented in red.
  • Figure 3: Results of experiment $\#9$ (and $\#10$). The trajectories in Figure \ref{['fig:P_reach_lower_Jackson_R_4']} correspond to Monte Carlo simulations taking samples from a distribution $\widetilde{p}_v\in\mathcal{P}_v$. The ones that satisfy the specification are presented in blue, while the ones that do not are presented in red.
  • Figure 4: Results of Experiment $\#11$. The trajectories in Figure \ref{['fig:P_reach_lower1_switched_linear2D']} correspond to Monte Carlo simulations taking samples from a Gaussian distribution $\widetilde{p}_v\in\mathcal{P}_v$ at the boundary of the ambiguity set. The ones that satisfy the specification are presented in blue, while the ones that do not are presented in red.
  • Figure 5: Results of Experiment $\#12$. The trajectories in Figure \ref{['fig:P_reach_upper2_switched_linear2D']} correspond to Monte Carlo simulations taking samples from a Gaussian distribution $\widetilde{p}_v\in\mathcal{P}_v$ at the boundary of the ambiguity set. The ones that satisfy the specification are presented in blue, while the ones that do not are presented in red.
  • ...and 2 more figures

Theorems & Definitions (34)

  • Definition 3.2: Switching Strategy
  • Definition 3.3: Reach-avoid probability
  • Remark 3.5
  • Definition 4.1: Robust MDP
  • Definition 4.2: IMDP
  • Definition 4.3: Strategy
  • Definition 4.4: Adversary
  • Definition 5.1
  • Proposition 5.2: Consistency of the Robust MDP Abstraction
  • Remark 5.3: Model Choice for the Abstraction
  • ...and 24 more