LLM-based MOFs Synthesis Condition Extraction using Few-Shot Demonstrations

TL;DR

The paper presents a few-shot in-context learning framework for extracting MOF synthesis conditions from the literature, powered by human–AI interactive data curation and retrieval-augmented generation to select 4–6 best demonstrations per paragraph. By incorporating external material knowledge through prompt engineering and post-processing, the approach achieves high extraction accuracy across multiple MOF datasets, outperforming zero-shot LLM baselines. The work demonstrates strong downstream impact in MOF design, showing real-world lab syntheses guided by LLM extractions can reach top-quality performance (e.g., BET surface area surpassing 91.1% of literature samples) and enables scalable, high-throughput literature processing via an online engine and database. Overall, the method reduces data-labeling costs compared with fine-tuning while enabling accurate synthesis-route extraction and material inference for structural and property design.

Abstract

The extraction of Metal-Organic Frameworks (MOFs) synthesis route from literature has been crucial for the logical MOFs design with desirable functionality. The recent advent of large language models (LLMs) provides disruptively new solution to this long-standing problem. While the latest researches mostly stick to primitive zero-shot LLMs lacking specialized material knowledge, we introduce in this work the few-shot LLM in-context learning paradigm. First, a human-AI interactive data curation approach is proposed to secure high-quality demonstrations. Second, an information retrieval algorithm is applied to pick and quantify few-shot demonstrations for each extraction. Over three datasets randomly sampled from nearly 90,000 well-defined MOFs, we conduct triple evaluations to validate our method. The synthesis extraction, structure inference, and material design performance of the proposed few-shot LLMs all significantly outplay zero-shot LLM and baseline methods. The lab-synthesized material guided by LLM surpasses 91.1% high-quality MOFs of the same class reported in the literature, on the key physical property of specific surface area.

Figures (26)

• Figure 1: Overview of our MOFs synthesis route extraction proposal using few-shot LLM in-context learning method. (A) The technical workflow is composed of multiple core components including data processing on material database, the RAG algorithm to select few-shot demonstrations, the LLM engine, and the multi-faceted material evaluation; (B) Statistics of MOFs datasets used in this work.
• Figure 2: Synthesis route extraction performance on the CSD-MOFs dataset. Key indicators (F1, ACC, Recall, Precision) are listed. Only literature with annotated ground-truth are included. (A) the proposed few-shot LLM (GPT-4 Turbo) with RAG algorithm; (B) zero-shot LLM as the baseline; (C) few-shot approach on 8 different LLMs; (D) zero-shot approach on 8 different LLMs; (E) confusion matrix definition for performance evaluation.
• Figure 3: Human-AI interactive data curation method. (A) the overall workflow to improve the data quality of few-shot examples; (B) the synthesis extraction performance by different data curation methods and varying number of shots. The 95% CI error bar by 5 repeated tests is only displayed on the human-AI curation plot to reduce visual clutter.
• Figure 4: The synthesis route extraction performance by RAG configurations. (A)(B) comparison of RAG algorithms, on F1 and ACC respectively; (C) comparison of different few-shot orders; (D) the impact of LLM prompt compositions. Error bars indicate 95% CI within 5 repeated tests.
• Figure 5: MOFs synthesis route extraction performance using example pool of varying sizes. (A) average F1 and its 95% CI on the 123-paragraph and 60-paragraph CSD-MOFs datasets ($K$=3 for both); (B) on the 123-paragraph dataset using different $K$-shots.