Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RLGT: A reinforcement learning framework for extremal graph theory

Ivan Damnjanović, Uroš Milivojević, Irena Đorđević, Dragan Stevanović

TL;DR

RLGT introduces a modular, Python-based reinforcement learning framework for extremal graph theory that unifies prior graph-building RL approaches. It supports directed/undirected graphs, loops, and arbitrary edge colors through a Graph class with eight formats, plus three RL environments (Linear, Global, Local) and three agents (Deep Cross-Entropy, REINFORCE, PPO) operating in batch mode. The framework is demonstrated on three applications—Laplacian spectral radius, graph energy vs. matching number, and Mostar index—showing counterexample discovery and insights, including replication of previous bounds and new counterexamples in some cases. RLGT emphasizes modularity, reproducibility, and extensibility, enabling researchers to explore and refine RL methods for extremal graph theory and to design new environments for nondeterministic tasks and broader problem classes.

Abstract

Reinforcement learning (RL) is a subfield of machine learning that focuses on developing models that can autonomously learn optimal decision-making strategies over time. In a recent pioneering paper, Wagner demonstrated how the Deep Cross-Entropy RL method can be applied to tackle various problems from extremal graph theory by reformulating them as combinatorial optimization problems. Subsequently, many researchers became interested in refining and extending the framework introduced by Wagner, thereby creating various RL environments specialized for graph theory. Moreover, a number of problems from extremal graph theory were solved through the use of RL. In particular, several inequalities concerning the Laplacian spectral radius of graphs were refuted, new lower bounds were obtained for certain Ramsey numbers, and contributions were made to the Turán-type extremal problem in which the forbidden structures are cycles of length three and four. Here, we present Reinforcement Learning for Graph Theory (RLGT), a novel RL framework that systematizes the previous work and provides support for both undirected and directed graphs, with or without loops, and with an arbitrary number of edge colors. The framework efficiently represents graphs and aims to facilitate future RL-based research in extremal graph theory through optimized computational performance and a clean and modular design.

RLGT: A reinforcement learning framework for extremal graph theory

TL;DR

RLGT introduces a modular, Python-based reinforcement learning framework for extremal graph theory that unifies prior graph-building RL approaches. It supports directed/undirected graphs, loops, and arbitrary edge colors through a Graph class with eight formats, plus three RL environments (Linear, Global, Local) and three agents (Deep Cross-Entropy, REINFORCE, PPO) operating in batch mode. The framework is demonstrated on three applications—Laplacian spectral radius, graph energy vs. matching number, and Mostar index—showing counterexample discovery and insights, including replication of previous bounds and new counterexamples in some cases. RLGT emphasizes modularity, reproducibility, and extensibility, enabling researchers to explore and refine RL methods for extremal graph theory and to design new environments for nondeterministic tasks and broader problem classes.

Abstract

Reinforcement learning (RL) is a subfield of machine learning that focuses on developing models that can autonomously learn optimal decision-making strategies over time. In a recent pioneering paper, Wagner demonstrated how the Deep Cross-Entropy RL method can be applied to tackle various problems from extremal graph theory by reformulating them as combinatorial optimization problems. Subsequently, many researchers became interested in refining and extending the framework introduced by Wagner, thereby creating various RL environments specialized for graph theory. Moreover, a number of problems from extremal graph theory were solved through the use of RL. In particular, several inequalities concerning the Laplacian spectral radius of graphs were refuted, new lower bounds were obtained for certain Ramsey numbers, and contributions were made to the Turán-type extremal problem in which the forbidden structures are cycles of length three and four. Here, we present Reinforcement Learning for Graph Theory (RLGT), a novel RL framework that systematizes the previous work and provides support for both undirected and directed graphs, with or without loops, and with an arbitrary number of edge colors. The framework efficiently represents graphs and aims to facilitate future RL-based research in extremal graph theory through optimized computational performance and a clean and modular design.
Paper Structure (16 sections, 1 theorem, 5 equations, 3 figures, 2 tables)

This paper contains 16 sections, 1 theorem, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph theory
  4. Reinforcement learning
  5. Framework overview
  6. Graphs and graph formats
  7. RL environments
  8. Linear environments
  9. Global environments
  10. Local environments
  11. Agents and RL methods
  12. Applications
  13. Laplacian spectral radius
  14. Graph energy and matching number
  15. Mostar index
  16. ...and 1 more sections

Key Result

Theorem 4.1

For any connected graph $G$ with $\Delta(G) \geqslant 6$, we have $\mathcal{E}(G) \leqslant 2 \nu(G) \sqrt{\Delta(G)}$.

Figures (3)

  • Figure 1: The best score versus step count plot for the Deep Cross-Entropy agent based disproof of Conjecture \ref{['agent_conj']}, and two counterexamples of order $16$.
  • Figure 2: The best score versus step count plot for the disproof of Conjecture \ref{['aaa_conj']}, and two counterexamples of order $14$.
  • Figure 3: The best score versus step count plot for the unsuccessful attempt to disprove Conjecture \ref{['mostar_conj']} for the case $n = 21$.

Theorems & Definitions (9)

  • Example 3.1
  • Example 3.2
  • Example 3.3
  • Example 3.4
  • Example 3.5
  • Conjecture 3.6: GheYaKaSte2024
  • Theorem 4.1: AkAlaAn2021
  • Conjecture 4.2: AkAlaAn2021
  • Conjecture 4.3: AliDo2021DoMaSkrTiZu2018