Generating Directed Graphs with Dual Attention and Asymmetric Encoding

TL;DR

This work introduces Directo, the first discrete flow matching–based generator for directed graphs, integrating direction-aware asymmetric positional encodings and a dual-attention graph transformer to capture both incoming and outgoing dependencies. By decoupling training and sampling, Directo achieves strong unconditional and conditional generation across synthetic DAGs and real-world digraphs, supported by a standardized benchmarking suite that evaluates validity, diversity, and distributional alignment. The approach demonstrates state-of-the-art performance on diverse graph types, including graphs with cycles and acyclic constraints, and showcases robustness through extensive ablations on dual attention and positional encodings. The work lays a solid foundation for directed graph generation with practical applicability to domains such as neural architecture search and scene graphs, while outlining clear avenues for scalability, conditioning, and constraint enforcement.

Abstract

Directed graphs naturally model systems with asymmetric, ordered relationships, essential to applications in biology, transportation, social networks, and visual understanding. Generating such graphs enables tasks such as simulation, data augmentation and novel instance discovery; however, directed graph generation remains underexplored. We identify two key factors limiting progress in this direction: first, modeling edge directionality introduces a substantially larger dependency space, making the underlying distribution harder to learn; second, the absence of standardized benchmarks hinders rigorous evaluation. Addressing the former requires more expressive models that are sensitive to directional topologies. We propose Directo, the first generative model for directed graphs built upon the discrete flow matching framework. Our approach combines: (i) principled positional encodings tailored to asymmetric pairwise relations, (ii) a dual-attention mechanism capturing both incoming and outgoing dependencies, and (iii) a robust, discrete generative framework. To support evaluation, we introduce a benchmark suite covering synthetic and real-world datasets. It shows that our method performs strongly across diverse settings and even competes with specialized models for particular classes, such as directed acyclic graphs. Our results highlight the effectiveness and generality of our approach, establishing a solid foundation for future research in directed graph generation.

Figures (17)

• Figure 1: (a) The learnable space increases drastically with the number of nodes for digraphs compared to undirected graphs harary1973enumeration, highlighting challenges in extending graph generative models to directed structures. (b) Overview of our generative model for directed graphs. During training, the dual attention transformer (denoising network), parameterized by $\theta$, is enhanced with asymmetric positional encoding, learning to reverse predictions using cross-entropy loss. During inference, we compute the rate matrix$R_t^{\theta}(G_t, G_{t+\Delta t})$, which governs the evolution of the generative process over finite intervals $\Delta t$, based on the model prediction $p^{\theta}(\cdot \mid G_t)$.
• Figure 2: Network architecture of Directo. We stack $L$ dual attention layers that account for both source-to-target and target-to-source information via cross-attention mechanisms. $\bm{X}$, $\bm{E}$ and $\bm{y}$ denote the stacked input node, edge, and global features. $\bm{X}'$, $\bm{E}'$ and $\bm{y}'$ are the output of the model, i.e., predicted clean node and edge distribution, and graph feature. FiLM perez2018film and PNA pooling layers corso2020principal are incorporated to enable flexible modulation between node, edge, and graph-level features. Full technical details are provided in Appendix \ref{['app:dualattention']}.
• Figure 3: Ablation results for dual attention (left) and positional encodings (right). Each plot shows results on the ER-DAG (left bars/lines) and SBM (right bars/lines) datasets. Better performance corresponds to V.U.N. bars and Ratio lines appearing closer to the top of each subplot.
• Figure 4: Sampling optimization curves for the synthetic datasets with 100 sampling steps and 5 sampling runs. We represent V.U.N. (blue) and MMD ratio (purple) and optimize for best trade-off for each of the three parameters individually.
• Figure 5: Sampling optimization curves for the synthetic datasets with 100 sampling steps and 5 sampling runs. We represent V.U.N. (blue) and MMD ratio (purple) and optimize for best trade-off for each of the three parameters individually.