Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating materials discovery using foundation model based In-context active learning

Jeffrey Hu, Rongzhi Dong, Ying Feng, Ming Hu, Jianjun Hu

Abstract

Active learning (AL) has emerged as a powerful paradigm for accelerating materials discovery by iteratively steering experiments toward the most promising candidates, reducing costly synthesis-and-characterization cycles. However, current AL relies predominantly on Gaussian Process (GP) and Random Forest (RF) surrogates with complementary limitations: GP underfits complex composition--property landscapes due to rigid kernel assumptions, while RF produces unreliable uncertainty estimates in small-data regimes, precisely where most materials datasets reside (with < 500 samples). Here we propose foudaiton model based In-Context Active Learning (ICAL), replacing conventional surrogates with TabPFN, a transformer-based foundation model pre-trained on millions of synthetic tasks to meta-learn a universal prior over tabular data. TabPFN performs principled Bayesian inference in a single forward pass without dataset-specific retraining, delivering well-calibrated predictive uncertainty where GP and RF fail most severely. Benchmarked against GP and RF across 10 materials datasets spanning copper alloy hardness and electrical conductivity, bulk metallic glass-forming ability, and crystal lattice thermal conductivity, TabPFN wins on 8 out of 10 datasets, achieving a mean saving of 52\% in extra experiments/evaluations relative to GP and 29.77% relative to RF. Cross-validation analysis confirms that TabPFN's advantage stems from superior uncertainty calibration,achieving the lowest Negative Log-Likelihood and Area Under the Sparsification Error curve among all surrogates. Our work demonstrates that a pre-trained foundation model can serve as a highly effective surrogate for accelerating active learning-based materials discovery.

Accelerating materials discovery using foundation model based In-context active learning

Abstract

Active learning (AL) has emerged as a powerful paradigm for accelerating materials discovery by iteratively steering experiments toward the most promising candidates, reducing costly synthesis-and-characterization cycles. However, current AL relies predominantly on Gaussian Process (GP) and Random Forest (RF) surrogates with complementary limitations: GP underfits complex composition--property landscapes due to rigid kernel assumptions, while RF produces unreliable uncertainty estimates in small-data regimes, precisely where most materials datasets reside (with < 500 samples). Here we propose foudaiton model based In-Context Active Learning (ICAL), replacing conventional surrogates with TabPFN, a transformer-based foundation model pre-trained on millions of synthetic tasks to meta-learn a universal prior over tabular data. TabPFN performs principled Bayesian inference in a single forward pass without dataset-specific retraining, delivering well-calibrated predictive uncertainty where GP and RF fail most severely. Benchmarked against GP and RF across 10 materials datasets spanning copper alloy hardness and electrical conductivity, bulk metallic glass-forming ability, and crystal lattice thermal conductivity, TabPFN wins on 8 out of 10 datasets, achieving a mean saving of 52\% in extra experiments/evaluations relative to GP and 29.77% relative to RF. Cross-validation analysis confirms that TabPFN's advantage stems from superior uncertainty calibration,achieving the lowest Negative Log-Likelihood and Area Under the Sparsification Error curve among all surrogates. Our work demonstrates that a pre-trained foundation model can serve as a highly effective surrogate for accelerating active learning-based materials discovery.
Paper Structure (17 sections, 4 figures, 6 tables)

This paper contains 17 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Pool based materials discovery using active learning
  4. In context active learning (ICAL) for materials discovery
  5. Normalized EI/UCB
  6. Hybrid Acquisition Functions:
  7. TabPFN: A Foundation Model Surrogate forr ALMD:
  8. Baseline surrogate model: Gaussian Process:
  9. Baseline surrogate model: Random Forest
  10. Benchmark Datasets
  11. Results
  12. Performance evaluation of ICAL in active-learning based materials discovery
  13. Importance of representation
  14. Mechanism Analysis: Explaining TabPFN's active-learning performance
  15. Discussion and Conclusion
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Pool-based active learning pipeline for discovering new materials. F: EI can be replaced with any other acquisition functions. G: k=1 is used for maximum sample efficiency. H: Top-1 task can be replaced with discovering Any of All of Top-K tasks.
  • Figure 2: Performance of ICAL on Cu_hardness dataset compared to GP and RF based AL. (a-b) Tabpfn against GP; (c-d): Tabpfn against RF.
  • Figure 3: Performance of ICAL on (a) (c) Cu_electric dataset and (b) (d) Glass_DS3 dataset compared to GP and RF based AL.
  • Figure 4: Performance of ICAL on LTC_conc dataset with element concentration features compared to GP and RF based AL.