Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fourier Transformers for Latent Crystallographic Diffusion and Generative Modeling

Jed A. Duersch, Elohan Veillon, Astrid Klipfel, Adlane Sayede, Zied Bouraoui

TL;DR

The paper tackles the challenge of generating crystalline materials under periodic boundary conditions and variable stoichiometries by moving the generative task to reciprocal space. It introduces a two-stage pipeline: a complex-valued transformer VAE that compresses a truncated Fourier representation of species-resolved unit-cell density, and a latent diffusion model that samples in this compact space before reconstructing coordinates. This reciprocal-space formulation makes space-group symmetries algebraic in the Fourier domain and supports multiplicities through the density, enabling flexible, symmetry-consistent generation. Empirical results on the LeMaterial dataset demonstrate recoverability of Fourier coefficients, stable VAE compression, and robust diffusion in the signal-dominant regime, with unconditional generation biased toward small-cell configurations but showing capacity for larger unit cells under conditioning. The approach offers a scalable alternative to coordinate-based crystal generators, with potential extensions to guided or size-aware diffusion to broaden generative coverage.

Abstract

The discovery of new crystalline materials calls for generative models that handle periodic boundary conditions, crystallographic symmetries, and physical constraints, while scaling to large and structurally diverse unit cells. We propose a reciprocal-space generative pipeline that represents crystals through a truncated Fourier transform of the species-resolved unit-cell density, rather than modeling atomic coordinates directly. This representation is periodicity-native, admits simple algebraic actions of space-group symmetries, and naturally supports variable atomic multiplicities during generation, addressing a common limitation of particle-based approaches. Using only nine Fourier basis functions per spatial dimension, our approach reconstructs unit cells containing up to 108 atoms per chemical species. We instantiate this pipeline with a transformer variational autoencoder over complex-valued Fourier coefficients, and a latent diffusion model that generates in the compressed latent space. We evaluate reconstruction and latent diffusion on the LeMaterial benchmark and compare unconditional generation against coordinate-based baselines in the small-cell regime ($\leq 16$ atoms per unit cell).

Fourier Transformers for Latent Crystallographic Diffusion and Generative Modeling

TL;DR

The paper tackles the challenge of generating crystalline materials under periodic boundary conditions and variable stoichiometries by moving the generative task to reciprocal space. It introduces a two-stage pipeline: a complex-valued transformer VAE that compresses a truncated Fourier representation of species-resolved unit-cell density, and a latent diffusion model that samples in this compact space before reconstructing coordinates. This reciprocal-space formulation makes space-group symmetries algebraic in the Fourier domain and supports multiplicities through the density, enabling flexible, symmetry-consistent generation. Empirical results on the LeMaterial dataset demonstrate recoverability of Fourier coefficients, stable VAE compression, and robust diffusion in the signal-dominant regime, with unconditional generation biased toward small-cell configurations but showing capacity for larger unit cells under conditioning. The approach offers a scalable alternative to coordinate-based crystal generators, with potential extensions to guided or size-aware diffusion to broaden generative coverage.

Abstract

The discovery of new crystalline materials calls for generative models that handle periodic boundary conditions, crystallographic symmetries, and physical constraints, while scaling to large and structurally diverse unit cells. We propose a reciprocal-space generative pipeline that represents crystals through a truncated Fourier transform of the species-resolved unit-cell density, rather than modeling atomic coordinates directly. This representation is periodicity-native, admits simple algebraic actions of space-group symmetries, and naturally supports variable atomic multiplicities during generation, addressing a common limitation of particle-based approaches. Using only nine Fourier basis functions per spatial dimension, our approach reconstructs unit cells containing up to 108 atoms per chemical species. We instantiate this pipeline with a transformer variational autoencoder over complex-valued Fourier coefficients, and a latent diffusion model that generates in the compressed latent space. We evaluate reconstruction and latent diffusion on the LeMaterial benchmark and compare unconditional generation against coordinate-based baselines in the small-cell regime ( atoms per unit cell).
Paper Structure (58 sections, 37 equations, 1 figure, 10 tables)

This paper contains 58 sections, 37 equations, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Overview and Crystal representation
  4. Representation
  5. Truncated Fourier Transform of Atomic Density
  6. Variational encoding and reverse diffusion
  7. Encoder--decoder architecture with auxiliary ladder
  8. Reverse diffusion in the auxiliary latent space
  9. Training Methodology
  10. Variational Auto-Encoder Training Objectives
  11. Reverse Diffusion Training
  12. Experiments
  13. Experimental protocol
  14. Representation fidelity and recoverability
  15. VAE reconstruction performance
  16. ...and 43 more sections

Figures (1)

  • Figure 1: Aux--Global--Fourier encoder--decoder with auxiliary ladder. The model separates tokens into a small auxiliary block, a single global structure token, and a large set of Fourier tokens. (a) The encoder distills information into a depth-aligned auxiliary ladder. (b) The decoder reconstructs structure and Fourier coefficients by injecting auxiliary ladder slices at matching depths.