Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amortized Network Intervention to Steer the Excitatory Point Processes

Zitao Song, Wendi Ren, Shuang Li

TL;DR

The paper tackles the problem of steering event flows generated by excitatory temporal point processes on evolving networks. It introduces Amortized Network Interventions (ANI), which partitions a large graph into subgraphs and learns a shared, permutation-equivalent policy that transfers to unseen regions; each subproblem is solved with a neural-ODE-based environment model and an efficient $H$-step model-based planning scheme that uses a mean-field approximation. A key contribution is the Permutation Equivalent Property (PE) and its Bi-Contrastive Embeddings (Bi-CME) with the Permutation Equivalent Metric (PEM), which enable robust cross-region policy transfer. Experimental results on synthetic data, real COVID datasets, and SUMO-based traffic data demonstrate improved generalization to unseen dynamics and effective network interventions under practical constraints. Overall, ANI provides a scalable approach to influence networked systems—such as epidemic spread and urban traffic—by learning transferable policies across regions with provable permutation-aware representations and efficient planning.

Abstract

Excitatory point processes (i.e., event flows) occurring over dynamic graphs (i.e., evolving topologies) provide a fine-grained model to capture how discrete events may spread over time and space. How to effectively steer the event flows by modifying the dynamic graph structures presents an interesting problem, motivated by curbing the spread of infectious diseases through strategically locking down cities to mitigating traffic congestion via traffic light optimization. To address the intricacies of planning and overcome the high dimensionality inherent to such decision-making problems, we design an Amortized Network Interventions (ANI) framework, allowing for the pooling of optimal policies from history and other contexts while ensuring a permutation equivalent property. This property enables efficient knowledge transfer and sharing across diverse contexts. Each task is solved by an H-step lookahead model-based reinforcement learning, where neural ODEs are introduced to model the dynamics of the excitatory point processes. Instead of simulating rollouts from the dynamics model, we derive an analytical mean-field approximation for the event flows given the dynamics, making the online planning more efficiently solvable. We empirically illustrate that this ANI approach substantially enhances policy learning for unseen dynamics and exhibits promising outcomes in steering event flows through network intervention using synthetic and real COVID datasets.

Amortized Network Intervention to Steer the Excitatory Point Processes

TL;DR

The paper tackles the problem of steering event flows generated by excitatory temporal point processes on evolving networks. It introduces Amortized Network Interventions (ANI), which partitions a large graph into subgraphs and learns a shared, permutation-equivalent policy that transfers to unseen regions; each subproblem is solved with a neural-ODE-based environment model and an efficient -step model-based planning scheme that uses a mean-field approximation. A key contribution is the Permutation Equivalent Property (PE) and its Bi-Contrastive Embeddings (Bi-CME) with the Permutation Equivalent Metric (PEM), which enable robust cross-region policy transfer. Experimental results on synthetic data, real COVID datasets, and SUMO-based traffic data demonstrate improved generalization to unseen dynamics and effective network interventions under practical constraints. Overall, ANI provides a scalable approach to influence networked systems—such as epidemic spread and urban traffic—by learning transferable policies across regions with provable permutation-aware representations and efficient planning.

Abstract

Excitatory point processes (i.e., event flows) occurring over dynamic graphs (i.e., evolving topologies) provide a fine-grained model to capture how discrete events may spread over time and space. How to effectively steer the event flows by modifying the dynamic graph structures presents an interesting problem, motivated by curbing the spread of infectious diseases through strategically locking down cities to mitigating traffic congestion via traffic light optimization. To address the intricacies of planning and overcome the high dimensionality inherent to such decision-making problems, we design an Amortized Network Interventions (ANI) framework, allowing for the pooling of optimal policies from history and other contexts while ensuring a permutation equivalent property. This property enables efficient knowledge transfer and sharing across diverse contexts. Each task is solved by an H-step lookahead model-based reinforcement learning, where neural ODEs are introduced to model the dynamics of the excitatory point processes. Instead of simulating rollouts from the dynamics model, we derive an analytical mean-field approximation for the event flows given the dynamics, making the online planning more efficiently solvable. We empirically illustrate that this ANI approach substantially enhances policy learning for unseen dynamics and exhibits promising outcomes in steering event flows through network intervention using synthetic and real COVID datasets.
Paper Structure (33 sections, 4 theorems, 24 equations, 18 figures, 4 tables, 1 algorithm)

This paper contains 33 sections, 4 theorems, 24 equations, 18 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Model-Based Reinforcement Learning
  4. Environment Model: Neural ODEs
  5. Online Planning: H-step lookahead with a learned model
  6. Making Large-Scale Problem Tractable: Amortized Policy
  7. Experimental Evaluation
  8. Network Intervention on Synthetic Data
  9. Evaluating Generalization on Covid Data
  10. Evaluating Generalization on Traffic Data
  11. Understanding Gains from PEM: Ablations and Visualizations
  12. Conclusions
  13. Permutation Equivalent Property
  14. Distinction Between Policy Equivalence and Policy Invariance
  15. Related work
  16. ...and 18 more sections

Key Result

Theorem 1

Let $J(t)$ and $\hat{J}(t)$ be the true and Mean Field Estimator for the cumulative cost. Let $\gamma=1$. Suppose we have $N$ nodes. When satisfying: Then we have: where $M_{n}=L_{\mathcal{T}_{n}}(L_0^{1/2} + L_0)$ and $\mathcal{T}_{n} : \mathbb{R}^{N \times d} \times \mathbb{R}^{N} \to \mathbb{R}^{d}$ is the composite transition function in Proposition prop:recursive_h.

Figures (18)

  • Figure 1: An illustration depicting the spread of a viral infection, where cities are represented as nodes and roads as edges.
  • Figure 2: Overview of the Method. The proposed Amortized Network Intervention contains three modules. The first module is to generate a latent node embedding $\mathbf{h}_{n}^{i}$ and evolve the latent states through the NJODE model. The second module learns a Permutation Equivalent Embedding (PEE) over the latent space $\mathbf{h}_{n}$ by a bi-contrastive loss function prepared for the downstream adaptation. The third module accesses the learned PEE from the second module and generates a permutation equivalent policy.
  • Figure 3: Cumulative intensity cost on synthetic datasets.
  • Figure 4: Generalization results of local community transformations on Covid data.
  • Figure 5: Two types of transformation of Covid data.
  • ...and 13 more figures

Theorems & Definitions (10)

  • Theorem 1: Error Bound of Mean Field Estimator
  • Definition 1: Permutation Equivalent Policy
  • Definition 2: Permutation Equivalent Metric, PEM
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Lemma 1
  • proof
  • proof