Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lang2Str: Two-Stage Crystal Structure Generation with LLMs and Continuous Flow Models

Cong Liu, Chengyue Gong, Zhenyu Liu, Jiale Zhao, Yuxuan Zhang

TL;DR

A two-stage generative framework that combines the strengths of large language models (LLMs) and flow-based models for flexible and precise material generation, and achieves competitive performance on material generation and crystal structure prediction tasks.

Abstract

Generative models hold great promise for accelerating material discovery but are often limited by their inflexible single-stage generative process in designing valid and diverse materials. To address this, we propose a two-stage generative framework, Lang2Str, that combines the strengths of large language models (LLMs) and flow-based models for flexible and precise material generation. Our method frames the generative process as a conditional generative task, where an LLM provides high-level conditions by generating descriptions of material unit cells' geometric layouts and properties. These descriptions, informed by the LLM's extensive background knowledge, ensure reasonable structure designs. A conditioned flow model then decodes these textual conditions into precise continuous coordinates and unit cell parameters. This staged approach combines the structured reasoning of LLMs and the distribution modeling capabilities of flow models. Experimental results show that our method achieves competitive performance on \textit{ab initio} material generation and crystal structure prediction tasks, with generated structures exhibiting closer alignment to ground truth in both geometry and energy levels, surpassing state-of-the-art models. The flexibility and modularity of our framework further enable fine-grained control over the generation process, potentially leading to more efficient and customizable material design.

Lang2Str: Two-Stage Crystal Structure Generation with LLMs and Continuous Flow Models

TL;DR

A two-stage generative framework that combines the strengths of large language models (LLMs) and flow-based models for flexible and precise material generation, and achieves competitive performance on material generation and crystal structure prediction tasks.

Abstract

Generative models hold great promise for accelerating material discovery but are often limited by their inflexible single-stage generative process in designing valid and diverse materials. To address this, we propose a two-stage generative framework, Lang2Str, that combines the strengths of large language models (LLMs) and flow-based models for flexible and precise material generation. Our method frames the generative process as a conditional generative task, where an LLM provides high-level conditions by generating descriptions of material unit cells' geometric layouts and properties. These descriptions, informed by the LLM's extensive background knowledge, ensure reasonable structure designs. A conditioned flow model then decodes these textual conditions into precise continuous coordinates and unit cell parameters. This staged approach combines the structured reasoning of LLMs and the distribution modeling capabilities of flow models. Experimental results show that our method achieves competitive performance on \textit{ab initio} material generation and crystal structure prediction tasks, with generated structures exhibiting closer alignment to ground truth in both geometry and energy levels, surpassing state-of-the-art models. The flexibility and modularity of our framework further enable fine-grained control over the generation process, potentially leading to more efficient and customizable material design.
Paper Structure (24 sections, 6 equations, 2 figures, 7 tables)

This paper contains 24 sections, 6 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Crystal and Crystal Structure Representations
  4. Space Group
  5. Task Definition
  6. Methods
  7. Flow Matching Models for Crystal Structures
  8. Lattice parameters $\mathbf{L}$
  9. Fraction coordinates $\mathbf{F}$
  10. Language Descriptor for Crystals
  11. Experiments
  12. Ab Initio Generation
  13. Dataset
  14. Results on Proxy Metrics
  15. Results on CHGNet-related Metrics
  16. ...and 9 more sections

Figures (2)

  • Figure 1: The generation of crystal structures.
  • Figure 2: Pipeline for Generating Crystal Structures from Textual Descriptions With Flows. This figure illustrates the overall pipeline for generating crystal structures using our framework at time step $t$. The process begins with retrieving space group information given chemical composition (e.g., GaTe) with CSPML. A fine-tuned LLM generates a textual description of the crystal structure given chemical composition and the retrieved space group, capturing its detailed geometric properties. The resulting text embedding is integrated with the node embeddings of the crystal structure through cross-attention layers, enabling efficient interactions between the textual and structural representations.