Machine Learning for the identification of phase-transitions in interacting agent-based systems: a Desai-Zwanzig example

TL;DR

The paper develops a data-driven coarse-graining workflow to identify phase transitions in a mean-field agent-based model, using Diffusion Maps to extract latent coordinates that align with the order parameter and a Y-shaped conformal autoencoder to disentangle parameter effects from state variables. A forward Euler–inspired residual neural network then learns a parameter-dependent ODE in the latent space, and symmetry-based transformations enable construction of a bifurcation diagram that captures the pitchfork-type transition. The approach yields a quantitatively accurate identification of the critical point and provides a versatile framework that can be extended to other ABMs lacking explicit low-dimensional closures. This work offers a path toward interpretable, data-driven reduced models for complex dynamical systems in the social and physical sciences.

Abstract

Deriving closed-form, analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM- the Desai-Zwanzig model in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates that facilitates, in our example, the identification of a single parameter-dependent ODE in these coordinates. We identify this ODE through a residual neural network inspired by a numerical integration scheme (forward Euler). We then use the identified ODE - enabled through an odd symmetry transformation - to construct the bifurcation diagram exhibiting the phase transition.

Figures (10)

• Figure 1: Schematic of the overall workflow. (I) Sample data from the Agent-Based Model (ABM) across multiple initial conditions and parameter values, (II) Compute histograms for each snapshot of the ABM. (III) Apply the Diffusion Maps algorithm on the computed histograms to discover a reduced latent embedding. (IV) Use a conformal autoencoder (AE) to find a conformal reparametrization of the latent space. (V) Identify a data-driven ODE in terms of the latent coordinate $\nu_2$ of the AE. (VI) Construct the bifurcation diagram (enabled via a symmetry transformation) in terms of the latent coordinate $\nu_2$.
• Figure 2: (a) The first two moments ($M_1$, $M_2$) estimated from the ABM simulations colored with the parameter value $\sigma$ at which they were obtained. (b-c) Diffusion Maps on histograms for a single value of the parameter $\sigma$: (b) One-to-one relation between leading histogram moment $M_1$ and leading Diffusion Maps coordinate $\psi_1$; (c) shows the residual ($r_{k}$) based on the local linear regression algorithm indicating that $\psi_1$ might be enough to parametrize the data (larger value of $r_k$). (d-f) Diffusion Maps on collected histograms from the ABM simulation across multiple values of the parameter $\sigma$: (d-e) show the non-harmonic diffusion map coordinates $(\psi_1, \psi_2)$ colored with $\sigma$ and $M_1$ respectively; (f) shows the residual ($r_{k}$) based on the local linear regression algorithm indicating that $\psi_1$ and $\psi_2$ might be enough to parametrize the data (larger values of $r_k$).
• Figure 3: (a) A schematic of the Y-shaped conformal autoencoder. The inputs to the network ($\psi_1$ and $\psi_2$) are shown as green nodes. The outputs of the autoencoder ($\hat{\psi}_1$ and $\hat{\psi}_2$) and the estimator are shown as red nodes. The latent variables ($\nu_1$ and $\nu_2$) are shown as light blue nodes. (b-c) The obtained latent coordinates $\nu_1,\nu_2$ colored with $\sigma$ and $M_1$ respectively. We can see that the $\sigma$ appears to vary only across $\nu_1$, compared to Fig. \ref{['fig:data_diffusion_maps']} (c) in which $\sigma$ varied across both $\psi_1, \psi_2$. Similarly, $M_1$ appears to vary only across $\nu_2$. (d) The parameter $\sigma$ is plotted against the latent coordinate $\nu_1$ indicating a strong dependence. (e) Contour lines representing level sets of $\nu_1$ and $\nu_2$ are plotted in the Diffusion Maps ($\psi_1$,$\psi_2$) providing a visual illustration of the obtained conformality.
• Figure 4: (a) A schematic of the forward Euler residual neural network. The state variable $\nu_2(t)$, the parameter $\sigma$, and the time step $h$ are inputs to the neural network that estimates the right-hand side $f_{\theta}$ of the ODE. The right-hand-side is then used to estimate the state variable $\hat{\nu_2}(t + h)$ by using a forward Euler step. (b) The predicted value for $\hat{\nu}_2(t +h)$ estimated from the Euler neural network is plotted against the true value (projected ABM trajectories in $\nu_2$ space) for values of the parameter $\sigma = \{0.57, 0.85, 1.11, 1.75, 1.9, 2.06, 2.25\}$ not included in the training set. (c) For three values, of the parameter not included in the training set we contrast generated paths from the ODE (solid lines) with paths simulated by the ABM (dashed lines) embedded in $\nu_2$. The colors black, blue (gray) and green (light-gray) corresponds to $\sigma = 1.11$, $\sigma = 1.75$ and $\sigma= 2.25$ respectively.
• Figure 5: (a) A representative bifurcation diagram constructed by the identified right-hand-side of the Euler neural network suggests a "perturbed" pitchfork. (b) A representative bifurcation diagram after applying equation \ref{['eq:symmetry']} to the identified right-hand-side shows a symmetric pitchfork.