Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the differentiability of the value function of switched linear systems under arbitrary and controlled switching

Guillaume O. Berger

TL;DR

The paper addresses differentiability of the optimal and worst-case value functions $J^\star$ and $J^\circ$ for switched linear systems. It proves Lipschitz continuity when the stage cost $c$ is Lipschitz and the joint spectral radius is below one, establishing a natural regularity baseline. The main contribution is a constructive demonstration that these value functions can be non-differentiable on dense subsets, even for smooth costs, by engineering a system whose optimal/worst-case switching induces an Interval Exchange Map with chaotic dynamics; this persists in higher dimensions and can be realized with rational matrices. The findings imply that exact computation of these value functions may require non-differentiable templates and highlight fundamental limits for optimization and reinforcement learning methods in this class of systems.

Abstract

This paper studies the differentiability of the value function of switched linear systems under arbitrary switching and controlled switching, referred to as worst-case and optimal value functions respectively. First, we show that the value functions are Lipschitz continuous, when the cost function is Lipschitz continuous. Then, as the central contribution of this work, we show with examples that each of these functions can be non-differentiable on dense subsets of the state space, even if the cost function is smooth and Lipschitz continuous. This has implications for optimal control and reinforcement learning since it implies that the exact computation of these value functions requires templates involving functions that are non-differentiable on dense subsets.

On the differentiability of the value function of switched linear systems under arbitrary and controlled switching

TL;DR

The paper addresses differentiability of the optimal and worst-case value functions and for switched linear systems. It proves Lipschitz continuity when the stage cost is Lipschitz and the joint spectral radius is below one, establishing a natural regularity baseline. The main contribution is a constructive demonstration that these value functions can be non-differentiable on dense subsets, even for smooth costs, by engineering a system whose optimal/worst-case switching induces an Interval Exchange Map with chaotic dynamics; this persists in higher dimensions and can be realized with rational matrices. The findings imply that exact computation of these value functions may require non-differentiable templates and highlight fundamental limits for optimization and reinforcement learning methods in this class of systems.

Abstract

This paper studies the differentiability of the value function of switched linear systems under arbitrary switching and controlled switching, referred to as worst-case and optimal value functions respectively. First, we show that the value functions are Lipschitz continuous, when the cost function is Lipschitz continuous. Then, as the central contribution of this work, we show with examples that each of these functions can be non-differentiable on dense subsets of the state space, even if the cost function is smooth and Lipschitz continuous. This has implications for optimal control and reinforcement learning since it implies that the exact computation of these value functions requires templates involving functions that are non-differentiable on dense subsets.

Paper Structure

This paper contains 10 sections, 23 theorems, 41 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Switched linear systems
  4. Optimal and worst-case value functions
  5. Lipschitz continuity of the value functions
  6. Possible non-differentiability of the value functions on dense subsets
  7. Proof of Theorem \ref{['thm:non-differentiable']}
  8. Case $n=2$
  9. Case $n>2$
  10. Conclusions

Key Result

Theorem 1

Let $\mathcal{A}\coloneqq\{A_i\}_{i=1}^m$ be a switched linear system and $\rho>\mathrm{jsr}(\mathcal{A})$. There is a norm $\lVert\cdot\rVert_*$ such that for all solutions $\xi$ of eq:sls-def and all $t\in\mathbb{N}$, it holds that $\lVert\xi(t)\rVert_*\leq\rho^t\lVert\xi(0)\rVert_*$.

Figures (2)

  • Figure 1: The function $\tilde{c}$ and a potential plot of $\tilde{J}^\star$ (left y-axis), along with $\Delta\tilde{J}^\star$ (right y-axis) for $\alpha=0.6$ and $\beta=\alpha-\pi/2$ ($\mu$ is the purple line).
  • Figure 2: The truncated backward trajectory $\{T^{-1}\nu,\ldots,T^{-15}\nu\}$ (black dots) of the map $T$ for $\alpha=0.6$, $\beta=\alpha-\pi/2$ and $\nu=0.1$. The blue arrows represent transitions by $T^{-1}$.

Theorems & Definitions (41)

  • Theorem 1: jungers2009thejoint
  • Proposition 1
  • Theorem 2
  • proof
  • Corollary 1
  • proof
  • Theorem 3
  • Lemma 1
  • Lemma 2
  • proof
  • ...and 31 more