Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous Uniqueness and Novelty Metrics for Generative Modeling of Inorganic Crystals

Masahiro Negishi, Hyunsoo Park, Kinga O. Mastej, Aron Walsh

TL;DR

The paper critiques the conventional structure-based distance $d_\mathrm{smat}$ for evaluating generative models of inorganic crystals and introduces two continuous distances, $d_\mathrm{magpie}$ for composition and $d_\mathrm{amd}$ for structure, with proven isometry invariance and Lipschitz continuity. It proves that discrete uniqueness under a pseudometric is permutation-invariant and demonstrates concrete theoretical advantages over $d_\mathrm{smat}$. Through experiments on MP20 across six models, continuous metrics reveal insights not captured by discrete distances, show strong speed advantages, and enable Pareto-based joint assessment of uniqueness and novelty. The results underscore the practical value of continuous crystal distances for fair model comparison and robust material discovery, and point to future directions in screening methods and alternative embeddings.

Abstract

To address pressing scientific challenges such as climate change, increasingly sophisticated generative artificial intelligence models are being developed that can efficiently sample the large chemical space of possible functional materials. These models can quickly sample new chemical compositions paired with crystal structures. They are typically evaluated using uniqueness and novelty metrics, which depend on a chosen crystal distance function. However, the most prevalent distance function has four limitations: it fails to quantify the degree of similarity between compounds, cannot distinguish compositional difference and structural difference, lacks Lipschitz continuity against shifts in atomic coordinates, and results in a uniqueness metric that is not invariant against the permutation of generated samples. In this work, we propose using two continuous distance functions to evaluate uniqueness and novelty, which theoretically overcome these limitations. Our experiments show that these distances reveal insights missed by traditional distance functions, providing a more reliable basis for evaluating and comparing generative models for inorganic crystals.

Continuous Uniqueness and Novelty Metrics for Generative Modeling of Inorganic Crystals

TL;DR

The paper critiques the conventional structure-based distance for evaluating generative models of inorganic crystals and introduces two continuous distances, for composition and for structure, with proven isometry invariance and Lipschitz continuity. It proves that discrete uniqueness under a pseudometric is permutation-invariant and demonstrates concrete theoretical advantages over . Through experiments on MP20 across six models, continuous metrics reveal insights not captured by discrete distances, show strong speed advantages, and enable Pareto-based joint assessment of uniqueness and novelty. The results underscore the practical value of continuous crystal distances for fair model comparison and robust material discovery, and point to future directions in screening methods and alternative embeddings.

Abstract

To address pressing scientific challenges such as climate change, increasingly sophisticated generative artificial intelligence models are being developed that can efficiently sample the large chemical space of possible functional materials. These models can quickly sample new chemical compositions paired with crystal structures. They are typically evaluated using uniqueness and novelty metrics, which depend on a chosen crystal distance function. However, the most prevalent distance function has four limitations: it fails to quantify the degree of similarity between compounds, cannot distinguish compositional difference and structural difference, lacks Lipschitz continuity against shifts in atomic coordinates, and results in a uniqueness metric that is not invariant against the permutation of generated samples. In this work, we propose using two continuous distance functions to evaluate uniqueness and novelty, which theoretically overcome these limitations. Our experiments show that these distances reveal insights missed by traditional distance functions, providing a more reliable basis for evaluating and comparing generative models for inorganic crystals.
Paper Structure (21 sections, 1 theorem, 12 equations, 6 figures, 6 tables)

This paper contains 21 sections, 1 theorem, 12 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Uniqueness and Novelty
  3. Uniqueness
  4. Novelty
  5. Theoretical Analysis of Crystal Distance Functions
  6. Experiments
  7. Conclusion
  8. Crystal Distance Functions
  9. Definitions
  10. $\bm{d_\mathrm{smat}}$
  11. $\bm{d_\mathrm{comp}}$
  12. $\bm{d_\mathrm{wyckoff}}$
  13. $\bm{d_\mathrm{magpie}}$
  14. $\bm{d_\mathrm{amd}}$
  15. Isometry Invariance and Lipschitz Continuity
  16. ...and 6 more sections

Key Result

Theorem 1

Assume $d_\mathrm{discrete}$ is a true mathematical pseudometric, i.e., satisfying the following properties for any $x$, $x'$, and $x"$: Then, the discrete uniqueness in Equation eq:uni defined by this $d_\mathrm{discrete}$ is invariant to the permutation of generated samples.

Figures (6)

  • Figure 1: wz-ZnO
  • Figure 2: wz-ZnO*
  • Figure 3: rs-ZnO
  • Figure 4: wz-GaN
  • Figure 5: Bi2Te3
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1: Discrete Uniqueness with Mathematical Pseudometric Is Permutation Invariant
  • proof