Data driven modeling for self-similar dynamics

TL;DR

A multiscale neural network framework that incorporates self-similarity as prior knowledge, facilitating the modeling ofSelf-similar dynamical systems, and allows us to extract scale-invariant kernels from the dynamics for modeling at any scale.

Abstract

Multiscale modeling of complex systems is crucial for understanding their intricacies. Data-driven multiscale modeling has emerged as a promising approach to tackle challenges associated with complex systems. On the other hand, self-similarity is prevalent in complex systems, hinting that large-scale complex systems can be modeled at a reduced cost. In this paper, we introduce a multiscale neural network framework that incorporates self-similarity as prior knowledge, facilitating the modeling of self-similar dynamical systems. For deterministic dynamics, our framework can discern whether the dynamics are self-similar. For uncertain dynamics, it can compare and determine which parameter set is closer to self-similarity. The framework allows us to extract scale-invariant kernels from the dynamics for modeling at any scale. Moreover, our method can identify the power law exponents in self-similar systems. Preliminary tests on the Ising model yielded critical exponents consistent with theoretical expectations, providing valuable insights for addressing critical phase transitions in non-equilibrium systems.

Figures (7)

• Figure 1: Framework
• Figure 2: (a).Diagram of for dynamics and coarse-graining in 2D lattice systems. The blue area represents the range of dynamical interactions, here is the nearest neighborhood interaction as an example. Blue dashed arrow means $\boldsymbol{x}^i$ or $\boldsymbol{y}^i$ evolves one step with dynamic $f^i$ or $F^i$. The red area represents a basic unit of microstate $\boldsymbol{\Tilde{X}}^{k}_{t}$ and it will be coarse-grained as a macrostate $\boldsymbol{y}^k_t$, displaying as a red point in the figure above. Both dynamics and coarse-graining operator could apply to any of other states and areas, which means the operators are homogeneous. (b).CNN model applied to our framework. We have two level neural network framework. The first level aims to capture the microscopic dynamics $f$ using the microstate which are presented blue in the bottom of the figure, while the second one represents the macroscopic state and dynamics $F$, which are red on the top. Both dynamics receive time-series data from their respective levels as input and produce predictions for future outcomes. As mentioned to in the main text, both the micro and macro dynamics share the same neural network structure and parameters, as depicted by $f(\theta_1)$ in the figure. Given the discrepancy in data dimensions between the micro and macro levels—typically with macro variables being fewer — it necessitates the construction of neural networks that exhibit spatio-temporal translational invariance. A coarse-graining learner $P(\theta_2)$,connects the two, facilitating the transformation from microscopic to macroscopic data. To prevent macrostates from falling into trivial state in the training process, we added a decoder $P(\theta_3)$ after the macroscopic prediction. This decoder constrains its output to closely resemble the microstates. After training, the decoder can be deactivated.
• Figure 3: Examples of CA dynamic renormalization based on self-similar constraints (rule = 60 and rule = 85). The first row is the result of rule 60. They are ground truth and renormalization results of three different scales with $S = T = 2,3,4$, respectively. The second row is the result of rule 85, and also are ground truth and renormalization results of three different scales with $S = T = 2,3,4$. We could see except $S = T = 3$, rule 60 can be renormalized as same rule in macro level, and rule 85 can be renormalized a self-similar dynamics only for $S = T = 3$.
• Figure 4: Visualization of dynamic prediction and consistency results for diffusion process. (a) shows the ground truth and prediction of microstate when $\Delta to \to 0$, respectively. Subplots are the enlarging version of the red square in order to get more clear visualization. (b) are visualization of consistency for dynamics when $\Delta t_0 \to 0$, same as CA before. We set $S=2$ and $T=4$ as example, while $S=3$ and $T=9$ also have the similar results, which we don't show here due to space constraints. (c) are the verification of the scaling relationship between time $t$ and average diffusion length $L$ in microstates and macrostates respectively.
• Figure 5: Comparison of different $\Delta t$ and quantitative results of dynamic consistency. (a) Testing the influence of different kernel sizes on dynamic accuracy. we found for the self-similar dynamic (left), kernel size = 5 and 7 could have the relative great results, while kernel size = 3 and 7 are better for none self-similar dynamic (right). So in the absence of any specific instructions, our experimental setup is kernel size = 7. (b). MSE for decoder module and the reconstruction error of self-similar dynamics is minimal. (c). Quantitative self-similar verification. We could found our framework is very effective for identifying self-similar dynamics. Each curve is the average result of three experiments, and the error bar is not obvious in the figure due to its small value