Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partial GFlowNet: Accelerating Convergence in Large State Spaces via Strategic Partitioning

Xuan Yu, Xu Wang, Rui Zhu, Yudong Zhang, Yang Wang

TL;DR

Experiments on several widely used datasets demonstrate that \modelname converges faster than existing works on large state spaces and not only generates candidates with higher rewards but also significantly improves their diversity.

Abstract

Generative Flow Networks (GFlowNets) have shown promising potential to generate high-scoring candidates with probability proportional to their rewards. As existing GFlowNets freely explore in state space, they encounter significant convergence challenges when scaling to large state spaces. Addressing this issue, this paper proposes to restrict the exploration of actor. A planner is introduced to partition the entire state space into overlapping partial state spaces. Given their limited size, these partial state spaces allow the actor to efficiently identify subregions with higher rewards. A heuristic strategy is introduced to switch partial regions thus preventing the actor from wasting time exploring fully explored or low-reward partial regions. By iteratively exploring these partial state spaces, the actor learns to converge towards the high-reward subregions within the entire state space. Experiments on several widely used datasets demonstrate that \modelname converges faster than existing works on large state spaces. Furthermore, \modelname not only generates candidates with higher rewards but also significantly improves their diversity.

Partial GFlowNet: Accelerating Convergence in Large State Spaces via Strategic Partitioning

TL;DR

Experiments on several widely used datasets demonstrate that \modelname converges faster than existing works on large state spaces and not only generates candidates with higher rewards but also significantly improves their diversity.

Abstract

Generative Flow Networks (GFlowNets) have shown promising potential to generate high-scoring candidates with probability proportional to their rewards. As existing GFlowNets freely explore in state space, they encounter significant convergence challenges when scaling to large state spaces. Addressing this issue, this paper proposes to restrict the exploration of actor. A planner is introduced to partition the entire state space into overlapping partial state spaces. Given their limited size, these partial state spaces allow the actor to efficiently identify subregions with higher rewards. A heuristic strategy is introduced to switch partial regions thus preventing the actor from wasting time exploring fully explored or low-reward partial regions. By iteratively exploring these partial state spaces, the actor learns to converge towards the high-reward subregions within the entire state space. Experiments on several widely used datasets demonstrate that \modelname converges faster than existing works on large state spaces. Furthermore, \modelname not only generates candidates with higher rewards but also significantly improves their diversity.
Paper Structure (29 sections, 14 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 29 sections, 14 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem formulation
  4. Partial Search
  5. Planner For Partial GFlowNet
  6. Selecting Module
  7. Updating Module
  8. Decision Module
  9. Partial Local Search
  10. Step A:Sampling
  11. Step B:Refining
  12. Step C:Training
  13. Step D: Updating
  14. Experiment
  15. Molecule Design
  16. ...and 14 more sections

Figures (9)

  • Figure 1: High-reward candidates can be rather sparse. By selecting partial space, the high-reward candidates become denser.
  • Figure 2: Overview of the proposed Partial GFlowNet framework.
  • Figure 3: Illustration of a single round of trajectory reconstruction. The procedure repeats $I$ times for each originally sampled trajectory.
  • Figure 4: Training curves comparison with respect to R_top$k$ and #Modes (R$>$7.5) on the task of Molecule Design.
  • Figure 5: Performance comparison with respect to #Modes on Sequence Generation task.
  • ...and 4 more figures