Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TFG-Flow: Training-free Guidance in Multimodal Generative Flow

Haowei Lin, Shanda Li, Haotian Ye, Yiming Yang, Stefano Ermon, Yitao Liang, Jianzhu Ma

TL;DR

TFG-Flow introduces a training-free guidance framework for multimodal flow models to steer generation towards target molecular properties without additional training. By combining discrete and continuous guidance within a flow-matching paradigm, it preserves flow marginals, ensures alignment with a time-independent predictor $f_c$, and maintains conditional independence of trajectories given data, with theoretical guarantees on guided velocity and rate matrices. The approach demonstrates strong empirical gains across QM9 quantum properties, structure fingerprints, and pocket-targeted drug design, outperforming many training-free baselines and narrowing gaps with conditional methods, especially when leveraging pre-trained predictors like UniMol. This work advances scalable, plug-and-play molecular design, enabling flexible target specification and efficient exploration of chemical space, with broad implications for drug discovery and multimodal generative modeling.

Abstract

Given an unconditional generative model and a predictor for a target property (e.g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training. As a highly efficient technique for steering generative models toward flexible outcomes, training-free guidance has gained increasing attention in diffusion models. However, existing methods only handle data in continuous spaces, while many scientific applications involve both continuous and discrete data (referred to as multimodality). Another emerging trend is the growing use of the simple and general flow matching framework in building generative foundation models, where guided generation remains under-explored. To address this, we introduce TFG-Flow, a novel training-free guidance method for multimodal generative flow. TFG-Flow addresses the curse-of-dimensionality while maintaining the property of unbiased sampling in guiding discrete variables. We validate TFG-Flow on four molecular design tasks and show that TFG-Flow has great potential in drug design by generating molecules with desired properties.

TFG-Flow: Training-free Guidance in Multimodal Generative Flow

TL;DR

TFG-Flow introduces a training-free guidance framework for multimodal flow models to steer generation towards target molecular properties without additional training. By combining discrete and continuous guidance within a flow-matching paradigm, it preserves flow marginals, ensures alignment with a time-independent predictor , and maintains conditional independence of trajectories given data, with theoretical guarantees on guided velocity and rate matrices. The approach demonstrates strong empirical gains across QM9 quantum properties, structure fingerprints, and pocket-targeted drug design, outperforming many training-free baselines and narrowing gaps with conditional methods, especially when leveraging pre-trained predictors like UniMol. This work advances scalable, plug-and-play molecular design, enabling flexible target specification and efficient exploration of chemical space, with broad implications for drug discovery and multimodal generative modeling.

Abstract

Given an unconditional generative model and a predictor for a target property (e.g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training. As a highly efficient technique for steering generative models toward flexible outcomes, training-free guidance has gained increasing attention in diffusion models. However, existing methods only handle data in continuous spaces, while many scientific applications involve both continuous and discrete data (referred to as multimodality). Another emerging trend is the growing use of the simple and general flow matching framework in building generative foundation models, where guided generation remains under-explored. To address this, we introduce TFG-Flow, a novel training-free guidance method for multimodal generative flow. TFG-Flow addresses the curse-of-dimensionality while maintaining the property of unbiased sampling in guiding discrete variables. We validate TFG-Flow on four molecular design tasks and show that TFG-Flow has great potential in drug design by generating molecules with desired properties.
Paper Structure (80 sections, 15 theorems, 49 equations, 2 figures, 9 tables, 1 algorithm)

This paper contains 80 sections, 15 theorems, 49 equations, 2 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. SO(3) Invariant 3D Molecule Modeling and Generation
  4. 3D molecular representations.
  5. Invariant probabilistic modeling for 3D molecules.
  6. Equivariant graph neural networks.
  7. multimodal Flow Model
  8. Conditional flow, velocity, and rate matrix.
  9. Deriving the unconditional flow.
  10. Methodology
  11. Problem setup.
  12. Constructing the Guided Flow
  13. Constructing appropriate guided flows.
  14. Finding the guided velocity and rate matrix.
  15. Discrete guidance: addressing curse of dimensionality
  16. ...and 65 more sections

Key Result

Theorem 3.1

Let ${\mathcal{G}}$ be the space of molecular representations and ${\mathcal{C}}$ be a finite set which includes all the values of our target property. Given a ${\mathcal{G}}$-valued process $\{{\bm{G}}_t\}_{t\in[0,1]}$ modeled by flow $\{p_{t}^{\mathrm{G}}({\bm{G}}_t)\}_{t\in[0,1]}$ and a function

Figures (2)

  • Figure 1: (a) Multimodal guided flow. The backward flow (blue arrow) is constructed with linear interpolation between observed data ${\bm{G}}_1$ and sampled noise ${\bm{G}}_0$; the forward flow (green arrows) is simulated by conditional velocity ${\bm{v}}_t({\bm{x}}_t^{(i)}|c)$and conditional rate matrix $R_t(a_t^{(i)},a_{t+\Delta t}^{(i)}|c)$. (b) The illustration of TFG-Flow. TFG-Flow guides the forward flow on each step ${\bm{G}}_t$, which consists of a gradient-based guidance for continuous part ${\bm{X}}_t$ and an importance sampling based guidance for discrete part ${\bm{a}}_t$; (c) Some examples of guidance targets.(d) Samples guided by TFG-Flow targeted at polarizability. The targeted value is at bottom of the samples.
  • Figure 2: Effect of varying hyperparameters ($N_{\mathrm{iter}}, \tau, \rho, K$). The line plot indicates the validity (left $y$-axis, higher values are better), while the bar plot shows the MAE (right $y$-axis, lower values are better).

Theorems & Definitions (27)

  • Theorem 3.1: Existence of the guided flow (informal)
  • Theorem 3.2: Guided velocity and rate matrix (informal)
  • Proposition 3.3
  • Theorem 3.4
  • Theorem 3.5: $\mathrm{SO}(3)$-invariance (proof in \ref{['app.proof-independence']})
  • Theorem B.1: Existence of the guided flow (formal version of \ref{['thm.independence']})
  • proof
  • Theorem B.2: Guided velocity
  • proof
  • Theorem B.3: Guided rate matrix
  • ...and 17 more