Connecting the geometry and dynamics of many-body complex systems with message passing neural operators

TL;DR

This work introduces ROMA, a framework that marries renormalization group ideas with neural operators to learn multiscale evolution for large-scale, multiscale, and noisy many-body systems. By integrating a neural renormalization precoder with a multiscale attention mechanism, ROMA learns local latent PDEs that approximate high-dimensional dynamics while remaining scalable to graphs with millions of nodes. Empirical results on Kuramoto and Burgers-like social dynamics show ROMA outperforming baselines in forecasting and effective-dynamics tasks, with clear evidence of positive transfer between tasks and insightful multiscale representations, alongside analysis of power-law scaling that reveals hierarchies in dynamics. The approach highlights the potential of combining geometric and spectral renormalization with physics-informed learning to yield scalable, interpretable models of complex systems, with broad applicability to physical, biological, and social networks.

Abstract

The relationship between scale transformations and dynamics established by renormalization group techniques is a cornerstone of modern physical theories, from fluid mechanics to elementary particle physics. Integrating renormalization group methods into neural operators for many-body complex systems could provide a foundational inductive bias for learning their effective dynamics, while also uncovering multiscale organization. We introduce a scalable AI framework, ROMA (Renormalized Operators with Multiscale Attention), for learning multiscale evolution operators of many-body complex systems. In particular, we develop a renormalization procedure based on neural analogs of the geometric and laplacian renormalization groups, which can be co-learned with neural operators. An attention mechanism is used to model multiscale interactions by connecting geometric representations of local subgraphs and dynamical operators. We apply this framework in challenging conditions: large systems of more than 1M nodes, long-range interactions, and noisy input-output data for two contrasting examples: Kuramoto oscillators and Burgers-like social dynamics. We demonstrate that the ROMA framework improves scalability and positive transfer between forecasting and effective dynamics tasks compared to state-of-the-art operator learning techniques, while also giving insight into multiscale interactions. Additionally, we investigate power law scaling in the number of model parameters, and demonstrate a departure from typical power law exponents in the presence of hierarchical and multiscale interactions.

Figures (5)

• Figure 1: Renormalized Operators with Multiscale Attention: The proposed model consists of message passing regions (pink) that conditions operators based on implicit representations of interactions, and vectorized regions (blue) that efficiently compute operators with vmap to forecast system evolution.
• Figure 1: Details of the Kuramoto Model (KM) and Burgers Dynamics (BD) datasets.
• Figure 2: Left: Average attention score from coarse-grained ($l=1$) to fine-grained ($l=0$) nodes in the first ROMA transformer block (KM-3M). Center: First 16 principle components of the attention scores. Right: Explained variance ratios of the first 16 principle components.
• Figure 3: Headwise attention statistics: (a-b) per-head fine to fine attention, (c-d) per-head coarse to fine attention, and (e-f) root mean squared (Frobenius) inter-head attention distance.
• Figure 4: Positional embedding similarity of scale/context embedding for KM-3M.