QE-Catalytic: A Graph-Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption

TL;DR

This work introduces QE-Catalytic, a multimodal base model that tightly couples an E(3)-equivariant graph Transformer with a large language model to predict relaxed adsorption energies and enable inverse design on catalytic surfaces. By sharing a graph–semantic latent space through graph–text alignment, QE-Catalytic integrates 3D structural details with structured textual descriptions, functioning effectively with or without precise coordinates. The model employs a Max–Min tanh-gated multitask loss to balance regression and generation objectives, and demonstrates superior performance on OC20 benchmarks compared to both text-based baselines and classical GNNs, while also enabling CIF-based inverse design and structure generation. The approach advances catalytic foundation modeling by enabling accurate energy prediction, discriminative handling of near-degenerate configurations, and a practical path toward structure generation when experimental or computational geometry is incomplete. The results suggest substantial practical impact for faster catalyst screening and materials design in real-world contexts, with future directions in kinetics integration, uncertainty estimation, and larger multimodal models.

Abstract

Adsorption energy is a key descriptor of catalytic reactivity. It is fundamentally defined as the difference between the relaxed total energy of the adsorbate-surface system and that of an appropriate reference state; therefore, the accuracy of relaxed-energy prediction directly determines the reliability of machine-learning-driven catalyst screening. E(3)-equivariant graph neural networks (GNNs) can natively operate on three-dimensional atomic coordinates under periodic boundary conditions and have demonstrated strong performance on such tasks. In contrast, language-model-based approaches, while enabling human-readable textual descriptions and reducing reliance on explicit graph -- thereby broadening applicability -- remain insufficient in both adsorption-configuration energy prediction accuracy and in distinguishing ``the same system with different configurations,'' even with graph-assisted pretraining in the style of GAP-CATBERTa. To this end, we propose QE-Catalytic, a multimodal framework that deeply couples a large language model (\textbf{Q}wen) with an E(3)-equivariant graph Transformer (\textbf{E}quiformer-V2), enabling unified support for adsorption-configuration property prediction and inverse design on complex catalytic surfaces. During prediction, QE-Catalytic jointly leverages three-dimensional structures and structured configuration text, and injects ``3D geometric information'' into the language channel via graph-text alignment, allowing it to function as a high-performance text-based predictor when precise coordinates are unavailable, while also autoregressively generating CIF files for target-energy-driven structure design and information completion. On OC20, QE-Catalytic reduces the MAE of relaxed adsorption energy from 0.713~eV to 0.486~eV, and consistently outperforms baseline models such as CatBERTa and GAP-CATBERTa across multiple evaluation protocols.

Figures (8)

• Figure 1: Comparison of training dynamics between the ordinary loss and MMTG-Loss. The ordinary loss decreases faster in the early stage but quickly enters a relatively high plateau. In contrast, MMTG-Loss exhibits a smoother overall descent, continues to decrease in later stages, and converges to a lower final loss level.
• Figure 2: Illustration of cross-modal alignment. The figure shows a similarity matrix between geometric embeddings and text embeddings, with the horizontal and vertical axes corresponding to sample indices. After multimodal alignment, a clear diagonal structure emerges in the similarity matrix, indicating that the geometric and textual representations of the same configuration are highly consistent in the latent space, whereas similarities between different configurations are substantially lower.
• Figure 3: An example of autocorrelation similarity heatmaps for multi-configuration samples under the same catalytic system. Each subfigure corresponds to a fixed adsorbate--catalyst pair; the horizontal and vertical axes index different configurations, and the color denotes cosine similarity of QE-Catalytic latent embeddings. It can be observed that even within the same system, the representation similarity between different configurations is substantially separated, indicating that the model can discriminate local geometric differences rather than collapsing "all configurations of the same system" into nearly identical representations.
• Figure 4: Performance gains from using LLM-derived configuration strings as inputs to QE-Catalytic. (a) Twelve representative examples selected from 66 adsorbate--catalyst pairs; colored dots indicate energies of different adsorption configurations under each pair. (b) Prediction Inclusion Ratio (PIR) over the 66 pairs, which quantifies the improvement in prediction accuracy after adding LLM-generated configuration strings ("config.") to the input.
• Figure 5: Ablation results for different output-head designs. The horizontal axis enumerates model variants (regression-only head, autoregressive-only head, and dual-head), and the vertical axis reports MAE and $R^2$. The dual-head design achieves slightly better performance on both metrics than either single-head variant, indicating that combining numerical regression with autoregressive generation benefits overall prediction performance.