Predicting Atomistic Transitions with Transformers

TL;DR

It is demonstrated how transformers can be trained to predict atomistic transitions in nano-clusters and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.

Abstract

Accurate knowledge of the atomistic transition pathways in materials and material surfaces is crucial for many material science problems. However, conventional simulation techniques used to find these transitions are extremely computationally intensive. Even with large-scale, accelerated material simulations, the computational cost constrains the applicable domain in practice. Machine learning models, with the potential to learn the complex emergent behaviors governing atomistic transitions as a fast surrogate model, have great promise to predict transitions with a vastly reduced computational cost. Here, we demonstrate how transformers can be trained to predict atomistic transitions in nano-clusters. We show how we evaluate physical validity of the predictions and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.

Figures (12)

• Figure 1: Three exemplar atomistic transitions originating from the same initial state. Red atoms represent atoms involved in the transition. Top row: The initial state, which is duplicated to show different colorings for the different atoms involved in each transition. Bottom row: Three different final states.
• Figure 2: Diagram of our transformer model. The architecture is nearly identical to the one introduced in vaswani2023attentionneed, though the input encoding scheme is modified to allow continuous input. Dotted lines indicate steps not involved in training.
• Figure 3: A bi-partite connectivity graph for an illustrative movement in a cluster of 7 atoms. Atom indices give arbitrary indexings.
• Figure 4: The "cumulative" radial distribution function, which aggregates the pair separation distances across all configurations. The position of the valley separating the first and second peak, 3.2 Å, is chosen as the connectivity threshold for creating a connectivity graph for each Pt cluster.
• Figure 5: Losses from individual trials in our hyperparameter scan. Top: Results our model not provided with hints during training (partial-position hinted). Bottom: Results for our model provided with hints during training (individual magnitude hinted). Plots were generated using Weights & Biases wandb.