Attention Mechanisms in Dynamical Systems: A Case Study with Predator-Prey Models

TL;DR

A simple linear attention model is trained on perturbed time-series data to reconstruct system trajectories and it is demonstrated that attention-based weighting can serve as a proxy for sensitivity analysis, capturing key phase-space properties without explicit knowledge of the system equations.

Abstract

Attention mechanisms are widely used in artificial intelligence to enhance performance and interpretability. In this paper, we investigate their utility in modeling classical dynamical systems -- specifically, a noisy predator-prey (Lotka-Volterra) system. We train a simple linear attention model on perturbed time-series data to reconstruct system trajectories. Remarkably, the learned attention weights align with the geometric structure of the Lyapunov function: high attention corresponds to flat regions (where perturbations have small effect), and low attention aligns with steep regions (where perturbations have large effect). We further demonstrate that attention-based weighting can serve as a proxy for sensitivity analysis, capturing key phase-space properties without explicit knowledge of the system equations. These results suggest a novel use of AI-derived attention for interpretable, data-driven analysis and control of nonlinear systems. For example our framework could support future work in biological modeling of circadian rhythms, and interpretable machine learning for dynamical environments.

Figures (4)

• Figure 1: Time-series plot of predator and prey populations. High-attention points (red dots prey, dark red triangles predators) identify states of maximum sensitivity within the trajectory, whereas low-attention points (blue dots prey, dark blue triangles predators) indicate states of minimal sensitivity. High-attention states primarily occur at peaks in prey population and points of rapid predator population increase, while low-attention states cluster around population troughs.
• Figure 2: Phase-space plot showing the predator-prey limit cycle explicitly. High-attention points (red) are concentrated in regions of high prey and moderate predator populations, representing critical system states sensitive to perturbation. Low-attention points (blue) occupy regions with lower prey populations and low predator populations, indicating relative insensitivity to perturbations.
• Figure 3: Phase-space plot of system trajectories following perturbations at one representative high-attention (red trajectory) and one low-attention (blue trajectory) point. The dashed green line represents the original, unperturbed limit cycle. Points at which perturbations were introduced are explicitly marked with larger red and blue crosses. Perturbation at the low-attention point drives the system substantially away from its original trajectory, potentially shifting the dynamics toward a larger-amplitude cycle. In contrast, the perturbation at the high-attention point results in a lesser deviation.
• Figure 4: The 3d surfave of the Lyapunov function