Reinforcement Learning Discovers Efficient Decentralized Graph Path Search Strategies

TL;DR

This work proposes a multi-agent approach for graph path search that successfully leverages both homophily and structural heterogeneity and shows that meaningful embeddings for graph navigation can be constructed using reward-driven learning.

Abstract

Graph path search is a classic computer science problem that has been recently approached with Reinforcement Learning (RL) due to its potential to outperform prior methods. Existing RL techniques typically assume a global view of the network, which is not suitable for large-scale, dynamic, and privacy-sensitive settings. An area of particular interest is search in social networks due to its numerous applications. Inspired by seminal work in experimental sociology, which showed that decentralized yet efficient search is possible in social networks, we frame the problem as a collaborative task between multiple agents equipped with a limited local view of the network. We propose a multi-agent approach for graph path search that successfully leverages both homophily and structural heterogeneity. Our experiments, carried out over synthetic and real-world social networks, demonstrate that our model significantly outperforms learned and heuristic baselines. Furthermore, our results show that meaningful embeddings for graph navigation can be constructed using reward-driven learning.

Figures (3)

• Figure 1: Mean Oracle Ratio obtained by the stochastic baselines on the validation dataset as a function of the temperature $\tau$ for varying values of $\beta$.
• Figure 2: Metric values obtained by the methods as a function of the synthetic graph density parameter $\beta$. GARDEN generally performs best, but it is notably surpassed by DistanceWalker in the truncation rate for high values of $\beta$.
• Figure 3: Visualization of the learned value function $v(u, u_\text{tgt})$ learned by GARDEN for each node $u$ (left) and preferability score $-\|\mathbf{x}_{u} - \mathbf{x}_{u_\text{tgt}}\|_{2} / \tau$ of the DistanceWalker baseline (right) for the social network graphs. The black arrows indicate the target node, while the color intensities of the other nodes are proportional to the value function learned by GARDEN (left) or baseline score (right). Concretely, dark red nodes indicate high proximity to the target, while dark blue nodes reflect low proximity. GARDEN recovers meaningful values for performing graph navigation, effectively leveraging proximity in both node attributes and topological structure.