Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Large Language Models for enzymatic reaction prediction and characterization

Lorenzo Di Fruscia, Jana Marie Weber

TL;DR

This work probes whether large language models (LLMs) can serve as biochemical reasoning engines for enzymatic reactions by evaluating EC-number prediction, forward synthesis, and retrosynthesis with Llama-3.1 8B/70B models using in-context learning and parameter-efficient fine-tuning (LoRA). By curating a BRENDA-derived dataset and enforcing leakage-free, group-based splits, the study shows that fine-tuned LLMs capture biochemical knowledge, with multitask training delivering notable gains in forward and retrosynthetic tasks. The 70B model achieves strong EC1 accuracy (~91.7%), and multitask learning enhances forward/retrosynthesis predictions, while still lagging behind specialized SOTA models in some metrics; results are particularly compelling in low-data regimes, where fine-tuning yields substantial gains over zero-shot performance. Overall, the findings underscore the potential of LoRA-enabled LLMs to accelerate biocatalysis design and enzyme-substrate prediction, while highlighting limitations related to EC hierarchy, data diversity, and computational cost that warrant further research.

Abstract

Predicting enzymatic reactions is crucial for applications in biocatalysis, metabolic engineering, and drug discovery, yet it remains a complex and resource-intensive task. Large Language Models (LLMs) have recently demonstrated remarkable success in various scientific domains, e.g., through their ability to generalize knowledge, reason over complex structures, and leverage in-context learning strategies. In this study, we systematically evaluate the capability of LLMs, particularly the Llama-3.1 family (8B and 70B), across three core biochemical tasks: Enzyme Commission number prediction, forward synthesis, and retrosynthesis. We compare single-task and multitask learning strategies, employing parameter-efficient fine-tuning via LoRA adapters. Additionally, we assess performance across different data regimes to explore their adaptability in low-data settings. Our results demonstrate that fine-tuned LLMs capture biochemical knowledge, with multitask learning enhancing forward- and retrosynthesis predictions by leveraging shared enzymatic information. We also identify key limitations, for example challenges in hierarchical EC classification schemes, highlighting areas for further improvement in LLM-driven biochemical modeling.

Leveraging Large Language Models for enzymatic reaction prediction and characterization

TL;DR

This work probes whether large language models (LLMs) can serve as biochemical reasoning engines for enzymatic reactions by evaluating EC-number prediction, forward synthesis, and retrosynthesis with Llama-3.1 8B/70B models using in-context learning and parameter-efficient fine-tuning (LoRA). By curating a BRENDA-derived dataset and enforcing leakage-free, group-based splits, the study shows that fine-tuned LLMs capture biochemical knowledge, with multitask training delivering notable gains in forward and retrosynthetic tasks. The 70B model achieves strong EC1 accuracy (~91.7%), and multitask learning enhances forward/retrosynthesis predictions, while still lagging behind specialized SOTA models in some metrics; results are particularly compelling in low-data regimes, where fine-tuning yields substantial gains over zero-shot performance. Overall, the findings underscore the potential of LoRA-enabled LLMs to accelerate biocatalysis design and enzyme-substrate prediction, while highlighting limitations related to EC hierarchy, data diversity, and computational cost that warrant further research.

Abstract

Predicting enzymatic reactions is crucial for applications in biocatalysis, metabolic engineering, and drug discovery, yet it remains a complex and resource-intensive task. Large Language Models (LLMs) have recently demonstrated remarkable success in various scientific domains, e.g., through their ability to generalize knowledge, reason over complex structures, and leverage in-context learning strategies. In this study, we systematically evaluate the capability of LLMs, particularly the Llama-3.1 family (8B and 70B), across three core biochemical tasks: Enzyme Commission number prediction, forward synthesis, and retrosynthesis. We compare single-task and multitask learning strategies, employing parameter-efficient fine-tuning via LoRA adapters. Additionally, we assess performance across different data regimes to explore their adaptability in low-data settings. Our results demonstrate that fine-tuned LLMs capture biochemical knowledge, with multitask learning enhancing forward- and retrosynthesis predictions by leveraging shared enzymatic information. We also identify key limitations, for example challenges in hierarchical EC classification schemes, highlighting areas for further improvement in LLM-driven biochemical modeling.
Paper Structure (33 sections, 1 equation, 16 figures, 6 tables)

This paper contains 33 sections, 1 equation, 16 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Tasks and dataset description
  4. Task selection
  5. Dataset preparation
  6. Data splitting
  7. LLM interaction and adaptation
  8. In-Context Learning
  9. Fine-tuning
  10. Model selection
  11. Evaluation metrics
  12. Fine-tuning setup
  13. Results and discussion
  14. Single-task fine-tuning
  15. EC prediction task
  16. ...and 18 more sections

Figures (16)

  • Figure 1: Distributions of samples across EC levels for the BRENDA dataset. The innermost layer represents the main class (EC1 digit), and the middle and outer layers represent levels EC2 and EC3 respectively. The label for enzyme class 7 (translocases) is not visible due to the limited data available (<20 samples).
  • Figure 2: Individual reactions sharing the same $\{product, EC\}$ or $\{substrate, EC\}$ pair are grouped together (here groups are numbered from 1 to 9, first row). The dataset is split into training and test set, while keeping each group intact. Within training and test, each group is assigned to one of the three tasks on a rotating basis to balance the splits. Groups are randomly shuffled at the beginning of the procedure, here we keep indices in order for visual clarity.
  • Figure 3: Distribution of reaction groups with repeating substrates and/or products. Unique reactions are included as elements with group size equal to 1. Group sizes with a number of counts $>10$ closely follow the required 70-30 split ratio between train and test set.
  • Figure 4: Example of a zero-shot prompt for the EC number prediction task. The model first receives a general prompt with instructions that inform it about the task to perform. The [TASK] here is EC number prediction, and the [OBJECTIVE] is to assign the 4 digits of the EC number given only reactants and product in SMILES notation. After that, the model receives the reaction SMILES from the user as a [REQUEST], and the model associates an EC number to it as the [RESPONSE].
  • Figure 5: Illustration of LoRA framework. The input vector x is passed through both the frozen weight matrix of the pretrained model, and the LoRA head. After both blocks process the input, the two representations are summed together to obtain a new representation $h$.
  • ...and 11 more figures