Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Materium: An Autoregressive Approach for Material Generation

Niklas Dobberstein, Jan Hamaekers

TL;DR

Materium introduces a discrete-token autoregressive framework for crystal-structure generation by encoding lattice parameters, fractional coordinates, elements, and oxidation states into a single sequence. The model is lightweight (≈43M parameters) and trains rapidly on a single GPU, enabling fast unconditional and multi-conditional generation for properties such as band gap, magnetic density, density, space group, and HHI. Empirical results show strong unconditional performance with high StructureMatcher match rates and energy proximity to relaxed structures, along with robust conditioned generation across several properties, though novelty lags behind some state-of-the-art diffusion methods. The approach offers a computationally efficient alternative to diffusion/flow models and provides room for reinforcement-learning fine-tuning to enhance output novelty and diversity.

Abstract

We present Materium: an autoregressive transformer for generating crystal structures that converts 3D material representations into token sequences. These sequences include elements with oxidation states, fractional coordinates and lattice parameters. Unlike diffusion approaches, which refine atomic positions iteratively through many denoising steps, Materium places atoms at precise fractional coordinates, enabling fast, scalable generation. With this design, the model can be trained in a few hours on a single GPU and generate samples much faster on GPUs and CPUs than diffusion-based approaches. The model was trained and evaluated using multiple properties as conditions, including fundamental properties, such as density and space group, as well as more practical targets, such as band gap and magnetic density. In both single and combined conditions, the model performs consistently well, producing candidates that align with the requested inputs.

Materium: An Autoregressive Approach for Material Generation

TL;DR

Materium introduces a discrete-token autoregressive framework for crystal-structure generation by encoding lattice parameters, fractional coordinates, elements, and oxidation states into a single sequence. The model is lightweight (≈43M parameters) and trains rapidly on a single GPU, enabling fast unconditional and multi-conditional generation for properties such as band gap, magnetic density, density, space group, and HHI. Empirical results show strong unconditional performance with high StructureMatcher match rates and energy proximity to relaxed structures, along with robust conditioned generation across several properties, though novelty lags behind some state-of-the-art diffusion methods. The approach offers a computationally efficient alternative to diffusion/flow models and provides room for reinforcement-learning fine-tuning to enhance output novelty and diversity.

Abstract

We present Materium: an autoregressive transformer for generating crystal structures that converts 3D material representations into token sequences. These sequences include elements with oxidation states, fractional coordinates and lattice parameters. Unlike diffusion approaches, which refine atomic positions iteratively through many denoising steps, Materium places atoms at precise fractional coordinates, enabling fast, scalable generation. With this design, the model can be trained in a few hours on a single GPU and generate samples much faster on GPUs and CPUs than diffusion-based approaches. The model was trained and evaluated using multiple properties as conditions, including fundamental properties, such as density and space group, as well as more practical targets, such as band gap and magnetic density. In both single and combined conditions, the model performs consistently well, producing candidates that align with the requested inputs.

Paper Structure

This paper contains 15 sections, 2 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Tokenization of Crystal Structures
  3. Crystal Definition
  4. Tokenization Process
  5. Model Architecture
  6. Training
  7. Dataset
  8. Training Procedure
  9. Evaluation
  10. Sensitivity to Token Ordering and Unconditional Generation
  11. Unconditional Generation $§M_{high}$
  12. Single-condition Comparison between Models
  13. Single-conditional Generation of $M_{high}$
  14. Multi Conditional Generation
  15. Conclusion

Figures (8)

  • Figure 1: Visualization of the tokenizer and model architecture used in Materium.
  • Figure 2: Space-group (left column) and reduced-formula (right column) results arranged by model (rows: $M_{xyz}$, $M_{low}$, $M_{high}$). The top 1 guess is blue, the top 2 guess is orange and the top 3 guess is green.
  • Figure 3: Comparison of the single-conditional densities for the $M_{xyz}$, $M_{low}$, and $M_{high}$ models. The plots illustrate the different predictive distributions learned by each model.
  • Figure 4: RMSD (top-left), band gap (top-right), magnetic density (bottom-left), and HHI score (bottom-right) for $M_{high}$.
  • Figure 5: Multi-conditional generation for the material's density, HHI score (divided by $1000$) and space group
  • ...and 3 more figures