Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Models

Jieyu Lu, Zhangde Song, Qiyuan Zhao, Yuanqi Du, Yirui Cao, Haojun Jia, Chenru Duan

TL;DR

This work finds that LLM-EO surpasses traditional GAs by leveraging the chemical knowledge of LLMs gained during their extensive pretraining, and introduces unparalleled flexibility into multi-objective optimizations, thereby circumventing the necessity for intricate mathematical formulations.

Abstract

Designing functional transition metal complexes (TMCs) faces challenges due to the vast search space of metals and ligands, requiring efficient optimization strategies. Traditional genetic algorithms (GAs) are commonly used, employing random mutations and crossovers driven by explicit mathematical objectives to explore this space. Transferring knowledge between different GA tasks, however, is difficult. We integrate large language models (LLMs) into the evolutionary optimization framework (LLM-EO) and apply it in both single- and multi-objective optimization for TMCs. We find that LLM-EO surpasses traditional GAs by leveraging the chemical knowledge of LLMs gained during their extensive pretraining. Remarkably, without supervised fine-tuning, LLMs utilize the full historical data from optimization processes, outperforming those focusing only on top-performing TMCs. LLM-EO successfully identifies eight of the top-20 TMCs with the largest HOMO-LUMO gaps by proposing only 200 candidates out of a 1.37 million TMCs space. Through prompt engineering using natural language, LLM-EO introduces unparalleled flexibility into multi-objective optimizations, thereby circumventing the necessity for intricate mathematical formulations. As generative models, LLMs can suggest new ligands and TMCs with unique properties by merging both internal knowledge and external chemistry data, thus combining the benefits of efficient optimization and molecular generation. With increasing potential of LLMs as pretrained foundational models and new post-training inference strategies, we foresee broad applications of LLM-based evolutionary optimization in chemistry and materials design.

Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Models

TL;DR

This work finds that LLM-EO surpasses traditional GAs by leveraging the chemical knowledge of LLMs gained during their extensive pretraining, and introduces unparalleled flexibility into multi-objective optimizations, thereby circumventing the necessity for intricate mathematical formulations.

Abstract

Designing functional transition metal complexes (TMCs) faces challenges due to the vast search space of metals and ligands, requiring efficient optimization strategies. Traditional genetic algorithms (GAs) are commonly used, employing random mutations and crossovers driven by explicit mathematical objectives to explore this space. Transferring knowledge between different GA tasks, however, is difficult. We integrate large language models (LLMs) into the evolutionary optimization framework (LLM-EO) and apply it in both single- and multi-objective optimization for TMCs. We find that LLM-EO surpasses traditional GAs by leveraging the chemical knowledge of LLMs gained during their extensive pretraining. Remarkably, without supervised fine-tuning, LLMs utilize the full historical data from optimization processes, outperforming those focusing only on top-performing TMCs. LLM-EO successfully identifies eight of the top-20 TMCs with the largest HOMO-LUMO gaps by proposing only 200 candidates out of a 1.37 million TMCs space. Through prompt engineering using natural language, LLM-EO introduces unparalleled flexibility into multi-objective optimizations, thereby circumventing the necessity for intricate mathematical formulations. As generative models, LLMs can suggest new ligands and TMCs with unique properties by merging both internal knowledge and external chemistry data, thus combining the benefits of efficient optimization and molecular generation. With increasing potential of LLMs as pretrained foundational models and new post-training inference strategies, we foresee broad applications of LLM-based evolutionary optimization in chemistry and materials design.

Paper Structure

This paper contains 10 sections, 5 figures.

Table of Contents

  1. Overview of LLM-EO.
  2. LLMs proposing high-quality TMCs in few-shot learning.
  3. Efficient optimization of HOMO-LUMO gap with LLM-EO.
  4. Flexible multi-objective optimization guided by natural language.
  5. Utilizing the generative power of LLMs for de novo TMC design.
  6. Genetic algorithms.
  7. Implementation Details of LLM-EO.
  8. Prompt Engineering.
  9. 1.37M square planar TMC space.
  10. Evaluating new TMCs.

Figures (5)

  • Figure 1: Overview of the design space of TMCs and LLM-EO.a. Construction of 1.37M space of square planar TMCs. The building blocks of TMCs consist of Pd in oxidation of II (i.e., metal) and a ligand pool with 25 neutral ligands and 25 ionic ligands. By assembling them in a square planar geometry with the constraint of total TMC charge being -1, 0, or 1, a space of 1.37M TMC is constructed. b. Workflow of LLM-EO. A prompt is engineered towards designing TMCs with both generic instructions (text in orange) and certain information, constraints, and objectives (text in blue). This prompt interacts with LLM (e.g., through API calls), which returns a new set of TMCs. These new TMCs are fed into the prompt with updated information, closing the loop of evolutionary optimization. Atoms are colored as follows: Pd for sky blue, C for gray, N for blue, O for red, P for orange, S for yellow, I for purple, and H for white. Purple dashed lines in TMCs represent dative bonds between Pd(II) and ligands.
  • Figure 2: Proposing new TMCs with few-shot LLMs.a. Box plot (notched) for the distribution of HOMO-LUMO gap of top-20 TMCs among the 200 TMCs proposed by different approaches, providing 20 TMCs as known. Random is for blue, GA for red, claude-3.5-sonnet for orange, o1-preview for green, o1-mini for purple, and gpt-4o for sky blue. b. Cumulative probability of HOMO-LUMO gaps of all TMCs proposed by various approaches and their overall distribution (box plot at top margin). c. Fraction of both valid and unique TMCs in 200 TMC proposals. d. Box plot (notched) for the distribution of HOMO-LUMO gap of top-20 TMCs among the 200 TMCs proposed, measured with a varying number of known TMCs provided in the prompt. Only the results for claude-3.5-sonnet (orange), o1-preview (green) are shown. The TMC with largest HOMO-LUMO gap for claude-3.5-sonnet and o1-preview is shown, correspondingly. The mean of random top-20 TMCs is shown with a blue dashed line. Atoms are colored as follows: Pd for sky blue, C for gray, N for blue, O for red, P for purple, F for green, and H for white. Purple dashed lines in TMCs represent dative bonds between Pd(II) and ligands.
  • Figure 3: Maximizing HOMO-LUMO gap with LLM-EO.a. Schematic of LLM-EO, with the option of keeping top-20 TMCs with largest HOMO-LUMO gaps in the prompt (green) or keeping all historical data (gray) during the evolutionary optimization. b. Cumulative probability of HOMO-LUMO gaps of all TMCs proposed by various approaches at iteration = 5 (left) and iteration = 20 (right) during the evolutionary optimization. Random sampling is used as blue, GA as red, o1-preview with only top-20 TMCs kept in green, and o1-preview with all historical data in gray. The overall distribution of each approach is shown as box plot at top margin. c. Box plot (notched) for the distribution of HOMO-LUMO gap of top-20 TMCs among the 200 TMCs proposed during the evolutionary optimization, providing 20 random TMCs as initial samples. d. Number of hits as top-20 TMCs with largest HOMO-LUMO gap among the 1.37M space versus the number of TMCs proposed during the evolutionary optimization. Solid lines are the average of three independent runs at different random seed, and their shedding corresponds to the standard deviation.
  • Figure 4: Multi-objective optimization with LLM-EO for Pareto frontier exploration (left), maximizing both HOMO-LUMO gap and polarisability ($\alpha$, middle), and maximizing HOMO-LUMO gap while keeping $\alpha$ < 1 eV. a. Polarisability versus HOMO-LUMO gap for random 400 TMCs (blue). The Pareto frontier (PF) for the 1.37 TMC space (red), the 3rd PF for the 1.37M TMC space (green), and the PF found by o1-preview with 400 LLM-EO exploration (gray), are shown. A solid step-wise line is drawn to shown the frontier area obtained by o1-preview in 400 TMCs exploration. b. Area under PF curves for the true 1st PF (red), 2rd PF (orange), 3rd PF (green), 4th PF (purple) for the 1.37M TMC space, and PF found by o1-preview (gray). c. Polarisability versus HOMO-LUMO gap for random 400 TMCs (blue) and top-200 TMCs with largest multiplication of HOMO-LUMO gap and polarisability for the 1.37M space (green) and 400 TMCs explored by o1-preview (gray). The Pareto frontier (PF) for the 1.37M TMC space (red) is also shown. d. Box plot (notched) for the distribution of HOMO-LUMO gap and polarisability multiplication of random TMCs (blue) and TMCs proposed by o1-preview (gray) during PF exploration, maximizing both gap and polarisability, and maximizing polarisability while keeping gap < 1 eV. e. Polarisability versus HOMO-LUMO gap for random 400 TMCs (blue) and top-200 TMCs with largest polarisability while gap < 1 eV for the 1.37M space (green) and 400 TMCs explored by o1-preview (gray). The Pareto frontier (PF) for the 1.37 TMC space (red) is also shown. f. Percentage of TMCs with HOMO-LUMO gap < 1 eV at various top-k polarisability by random sampling (blue), o1-preview at single-objective optimization for only maximizing polarisability (black), and o1-preview at multi-objective optimization for maximizing polarisability while requiring HOMO-LUMO gap < 1 eV (gray). Select TMCs are shown for each case as insets. Atoms are colored as follows: Pd for sky blue, C for gray, N for blue, O for red, P for orange, F for green, and H for white. Purple dashed lines in TMCs represent dative bonds between Pd(II) and ligands.
  • Figure 5: Generating new ligands and TMCs with LLMs.a. Workflow for assembling new TMC space by asking LLMs to propose new ligands outside of the original ligand pool. b. Workflow for direct iterative TMC optimization by asking LLMs to propose new TMCs that are made by new ligands outside of the original ligand pool. c. Cumulative probability of polarisability of TMCs proposed by various approaches. Random sampling is used as blue, using 10 ligands generated by o1-preview in one shot in red, LLM-EO by o1-preview for generating new TMCs iteratively in orange. d. Box plot (notched) for the distribution of polarisability for TMCs explored by various approaches. The best TMC for each approach is shown. e. Polarisability versus HOMO-LUMO gap for random 400 TMCs (blue), TMCs constructed by 10 ligands generated by o1-preview in one shot (red), and TMCs generated by LLM-EO using o1-preview iteratively (orange). Box plots for the distribution of each property are shown at margins. f. Box plot (notched) for the distribution of the product of HOMO-LUMO-gap and polarisability (that is, $\alpha$) for TMCs explored by various approaches. The best TMC for each approach is shown. Atoms are colored as follows: Pd for sky blue, C for gray, N for blue, O for red, F for green, P for orange, S for yellow, and H for white. Purple dashed lines in TMCs represent dative bonds between Pd(II) and ligands.