Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Sequential Decision Making over Expanding Networks with Graph Filtering

Zhan Gao, Bishwadeep Das, Elvin Isufi

Abstract

Graph filters leverage topological information to process networked data with existing methods mainly studying fixed graphs, ignoring that graphs often expand as nodes continually attach with an unknown pattern. The latter requires developing filter-based decision-making paradigms that take evolution and uncertainty into account. Existing approaches rely on either pre-designed filters or online learning, limited to a myopic view considering only past or present information. To account for future impacts, we propose a stochastic sequential decision-making framework for filtering networked data with a policy that adapts filtering to expanding graphs. By representing filter shifts as agents, we model the filter as a multi-agent system and train the policy following multi-agent reinforcement learning. This accounts for long-term rewards and captures expansion dynamics through sequential decision-making. Moreover, we develop a context-aware graph neural network to parameterize the policy, which tunes filter parameters based on information of both the graph and agents. Experiments on synthetic and real datasets from cold-start recommendation to COVID prediction highlight the benefits of using a sequential decision-making perspective over batch and online filtering alternatives.

Stochastic Sequential Decision Making over Expanding Networks with Graph Filtering

Abstract

Graph filters leverage topological information to process networked data with existing methods mainly studying fixed graphs, ignoring that graphs often expand as nodes continually attach with an unknown pattern. The latter requires developing filter-based decision-making paradigms that take evolution and uncertainty into account. Existing approaches rely on either pre-designed filters or online learning, limited to a myopic view considering only past or present information. To account for future impacts, we propose a stochastic sequential decision-making framework for filtering networked data with a policy that adapts filtering to expanding graphs. By representing filter shifts as agents, we model the filter as a multi-agent system and train the policy following multi-agent reinforcement learning. This accounts for long-term rewards and captures expansion dynamics through sequential decision-making. Moreover, we develop a context-aware graph neural network to parameterize the policy, which tunes filter parameters based on information of both the graph and agents. Experiments on synthetic and real datasets from cold-start recommendation to COVID prediction highlight the benefits of using a sequential decision-making perspective over batch and online filtering alternatives.
Paper Structure (9 sections, 6 equations, 2 figures)

This paper contains 9 sections, 6 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Filtering over Expanding Graphs
  4. Stochastic Sequential Decision Making
  5. Graph Multi-Agent Reinforcement Learning
  6. Multi-Agent Reinforcement Learning
  7. Context-Aware Graph Neural Networks
  8. Numerical results
  9. Conclusions

Figures (2)

  • Figure 1: (Left) Framework of Graph Multi-Agent Reinforcement Learning. (Right) Context-aware Graph Neural Network.
  • Figure 2: (a) Training convergence of G-MARL. (b) Performance comparison for synthetic data. (c) Generalization of G-MARL to unseen scenarios. (d) Performance comparison for recommender systems. (e) Performance comparison for COVID predication.