Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach

TL;DR

An Equation/Variable free machine learning (EVFML) framework for the control of the collective dynamics of complex/multiscale systems modelled via microscopic/agent-based simulators is presented, demonstrating that the scheme is robust against numerical approximation errors and modelling uncertainty.

Abstract

We present an Equation/Variable free machine learning (EVFML) framework for the control of the collective dynamics of complex/multiscale systems modelled via microscopic/agent-based simulators. The approach obviates the need for construction of surrogate, reduced-order models.~The proposed implementation consists of three steps: (A) from high-dimensional agent-based simulations, machine learning (in particular, non-linear manifold learning (Diffusion Maps (DMs)) helps identify a set of coarse-grained variables that parametrize the low-dimensional manifold on which the emergent/collective dynamics evolve. The out-of-sample extension and pre-image problems, i.e. the construction of non-linear mappings from the high-dimensional input space to the low-dimensional manifold and back, are solved by coupling DMs with the Nystrom extension and Geometric Harmonics, respectively; (B) having identified the manifold and its coordinates, we exploit the Equation-free approach to perform numerical bifurcation analysis of the emergent dynamics; then (C) based on the previous steps, we design data-driven embedded wash-out controllers that drive the agent-based simulators to their intrinsic, imprecisely known, emergent open-loop unstable steady-states, thus demonstrating that the scheme is robust against numerical approximation errors and modelling uncertainty.~The efficiency of the framework is illustrated by controlling emergent unstable (i) traveling waves of a deterministic agent-based model of traffic dynamics, and (ii) equilibria of a stochastic financial market agent model with mimesis.

Figures (11)

• Figure 1: Schematic of the proposed data-driven EVFML framework, based on machine learning, for the control of the emergent/collective dynamics of complex systems modelled via agent-based simulators. The approach is deployed in three main steps: (A) Discovery of the low-dimensional manifold $\mathcal{M}$ and its coordinates using non-linear manifold learning and in particular Diffusion Maps (DMs), from a high-dimensional point cloud, (B) Equation-free numerical bifurcation analysis on $\mathcal{M}$, (C) Repetition of (A) around the coarse-grained steady-states of interest over the control parameter space and "on demand" design of wash-out controllers for the emergent dynamics.
• Figure 2: Schematic representation of the out-of-sample extension problem (blue dots and arrows) via Nyström method and the pre-image problem via GHs (red dots and arrows) at panel (a) and via k-NN at panel (b). The black dots denote observations in the ambient space $\mathbb{R}^N$ and their representations on the DMs and "double" DMs spaces $\mathbb{R}^D$ and $\mathbb{R}^K$, respectively. The Nyström method is performed for obtaining the image $\mathcal{R}(\mathbf{x}^n_l)$ of the new point $\mathbf{x}^n_l$. The reconstruction with GHs (top panel) and with k-NN (bottom panel) is performed for obtaining the pre-image $\mathcal{L}(\mathbf{y}^n_l)$ of a new point $\mathbf{y}^n_l$. As described in Section \ref{['subsub:Lift']} for the GHs extension, projection to the "double" DMs space is first performed for defining a new basis in $\mathbb{R}^K$ through which one obtains the reconstructed state in $\mathbb{R}^N$.
• Figure 3: A schematic of the Equation-free multiscale framework.
• Figure 4: Agent-based traffic model. (a) The first 20 eigenvalues of the DMs coordinates over the bifurcation parameter space, and (b) the corresponding first two DMs coordinates, colored by the standard deviation $\sigma$. (c) Coarse-grained bifurcation diagram constructed through the EVFML framework by considering the first 2 leading DMs coordinates as coarse-grained variables; for lifting, both GHs and k-NN were employed. The analytical derived bifurcation diagram is superimposed; solid (dotted) lines denote stable (unstable) steady-states. The non-zero solution branch corresponds to traveling waves solutions, while the zero solution branch corresponds to free-flow solutions. (d) Travelling wave profiles at selected points ($P_1$-$P_4$); notice the change in the scale of the y-axis. (e),(f) EVFML Control of the agent-based traffic dynamics. Implementation of the EVFML controller for stabilizing the coarse-grained unstable traveling wave at $v_0 =1.0099$, using $h$ as control variable. (e) Coarse-grained open-loop and closed-loop responses in $\sigma$ coordinates, and (f) the response of the control variable $h(t)$. The EVFML controller drives and stabilizes the emergent dynamics to the actual open loop unstable steady-state. The results obtained assuming the prior knowledge of the coarse-grained variable ($\sigma$) are also given for comparison reasons.
• Figure 5: Stochastic agent-based model of a financial market with mimesis. (a) The first 20 eigenvalues of the DMs coordinates over the bifurcation parameter space, and (b) the corresponding first DMs coordinate, plotted against the mean value of the distribution $\overline{X}$. (c) Coarse-grained bifurcation diagrams constructed through the EVFML framework by considering the first DMs coordinate $\mathbf{y}_1$ as the coarse-grained variable; for lifting both GHs and k-NN were considered. Solid (dotted) lines denote stable (unstable) steady-states. For comparison purposes, we also depict the coarse-grained bifurcation diagram constructed using $\overline{X}$ (with black color) as coarse-grained variable that was discovered to be in a one-to-one correspondence with the first coordinate $\mathbf{y}_1$ of the DMs analysis, as well as the one constructed using the ICDF of the microscopic distribution and $R^{\pm}$ as coarse-grained variables (siettos2012equation) on the basis of $50,000$ agents (with blue color). (d) Density profiles of the distribution at selected points ($P_1$-$P_3$) along the bifurcation branch. (e),(f) Control of the stochastic agent-based financial market dynamics. Implementation of the wash-out embedded controller for stabilizing the coarse-grained unstable equilibrium at $g =41$, using $\nu_{ex}^-$ as control variable. (e) Coarse-grained open-loop and closed-loop responses with respect to $\overline{X}$, and (f) the responses of the control variable $\nu_{ex}^-(t)$. The controller drives and stabilizes the emergent dynamics to the actual open loop unstable steady-state. Due to stochasticity, mean values (solid lines) and 95% CI (shaded regions around them) are also reported based on $2,000$ runs.

Theorems & Definitions (2)

• Theorem 2.1
• proof