Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Pontryagin Maximum Principle for agent-based models with convex state space

Stefano Almi, Riccardo Durastanti, Francesco Solombrino

TL;DR

The paper develops a Pontryagin Maximum Principle for agent-based systems whose state evolves in a convex subset $C$ of a Banach space $E$, including mean-field limits in Wasserstein spaces. It introduces $C$-differentiability and a specialized linearisation to overcome the lack of linear structure, and derives adjoint equations and Hamiltonian maximality conditions. The framework is extended to finite-particle systems and to the Wasserstein space with a Wasserstein gradient $\nabla_\mu A$, culminating in a PMP that couples a maximality condition with a Wasserstein Hamiltonian flow on $C \times E_C^*$. Model examples, namely leader-follower dynamics and entropy-regularised replicator dynamics, illustrate the applicability and pave the way for numerical and multi-population control insights.

Abstract

We derive a first order optimality condition for a class of agent-based systems, as well as for their mean-field counterpart. A relevant difficulty of our analysis is that the state equation is formulated on possibly infinite-dimensional convex subsets of Banach spaces. This is a typical feature of many problems in multi-population dynamics, where a convex set of probability measures may account for the population, the degree of influence or the strategy attached to each agent. Due to the lack of a linear structure and of local compactness, the usual tools of needle variations and linearisation procedures used to derive Pontryagin type conditions have to be generalised to the setting at hand. This is done by considering suitable notions of differentials and by a careful inspection of the underlying functional structures.

A Pontryagin Maximum Principle for agent-based models with convex state space

TL;DR

The paper develops a Pontryagin Maximum Principle for agent-based systems whose state evolves in a convex subset of a Banach space , including mean-field limits in Wasserstein spaces. It introduces -differentiability and a specialized linearisation to overcome the lack of linear structure, and derives adjoint equations and Hamiltonian maximality conditions. The framework is extended to finite-particle systems and to the Wasserstein space with a Wasserstein gradient , culminating in a PMP that couples a maximality condition with a Wasserstein Hamiltonian flow on . Model examples, namely leader-follower dynamics and entropy-regularised replicator dynamics, illustrate the applicability and pave the way for numerical and multi-population control insights.

Abstract

We derive a first order optimality condition for a class of agent-based systems, as well as for their mean-field counterpart. A relevant difficulty of our analysis is that the state equation is formulated on possibly infinite-dimensional convex subsets of Banach spaces. This is a typical feature of many problems in multi-population dynamics, where a convex set of probability measures may account for the population, the degree of influence or the strategy attached to each agent. Due to the lack of a linear structure and of local compactness, the usual tools of needle variations and linearisation procedures used to derive Pontryagin type conditions have to be generalised to the setting at hand. This is done by considering suitable notions of differentials and by a careful inspection of the underlying functional structures.
Paper Structure (8 sections, 14 theorems, 269 equations)

This paper contains 8 sections, 14 theorems, 269 equations.

Table of Contents

  1. Introduction
  2. Preliminaries and notation
  3. Pontryagin maximum principle for optimal control problems in convex spaces
  4. A generalization to finite particle control problems
  5. Pontryagin maximum principle in the Wasserstein space of convex spaces
  6. Model examples
  7. Leader-follower dynamics
  8. Replicator-type models with entropy regularisation

Key Result

Lemma 3.1

Let $A$ satisfy $(vi)$. Then the map $\mathrm{D} A \colon C \to \mathcal{L}(E_C;E_C)$ with $\mathrm{D} A(c):=\mathrm{D}_cA$ is single valued. Moreover, if $(i)$ holds, $\mathrm{D}_{c(t)}A:=B_{(t,c(t),u(t))}\in L^\infty([0,T];\mathcal{L}(E_C;E_C))$ for every $c\in AC([0,T];C)$ and $u\in\mathcal{U}$.

Theorems & Definitions (31)

  • Definition 2.1
  • Lemma 3.1
  • proof
  • Remark 3.2
  • Theorem 3.3
  • proof
  • Theorem 3.4
  • proof
  • Theorem 3.5
  • Remark 4.1
  • ...and 21 more