Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Empowering LLMs for Structure-Based Drug Design via Exploration-Augmented Latent Inference

Xuanning Hu, Anchen Li, Qianli Xing, Jinglong Ji, Hao Tuo, Bo Yang

TL;DR

This paper tackles structure-based drug design by addressing the limitations of LLMs in understanding protein structures and guiding targeted molecule generation. It introduces ELILLM, a three-stage framework that reinterprets LLM generation as encoding, exploration in latent space via Bayesian optimization, and knowledge-guided decoding to produce chemically valid ligands. A Deep Kernel Gaussian Process surrogate with a position-aware aggregation guides latent-space exploration, while domain-informed decoding constrains outputs to plausible SMILES, resulting in higher predicted binding affinities on CrossDocked2020 and competitive performance against seven baselines. The approach demonstrates that explicit latent-space exploration can augment pretrained LLMs for domain-specific design tasks, offering interpretability, controllability, and compatibility with existing generation models. The work has practical implications for accelerating SBDD workflows and motivating broader use of uncertainty-aware latent inference in scientific AI.

Abstract

Large Language Models (LLMs) possess strong representation and reasoning capabilities, but their application to structure-based drug design (SBDD) is limited by insufficient understanding of protein structures and unpredictable molecular generation. To address these challenges, we propose Exploration-Augmented Latent Inference for LLMs (ELILLM), a framework that reinterprets the LLM generation process as an encoding, latent space exploration, and decoding workflow. ELILLM explicitly explores portions of the design problem beyond the model's current knowledge while using a decoding module to handle familiar regions, generating chemically valid and synthetically reasonable molecules. In our implementation, Bayesian optimization guides the systematic exploration of latent embeddings, and a position-aware surrogate model efficiently predicts binding affinity distributions to inform the search. Knowledge-guided decoding further reduces randomness and effectively imposes chemical validity constraints. We demonstrate ELILLM on the CrossDocked2020 benchmark, showing strong controlled exploration and high binding affinity scores compared with seven baseline methods. These results demonstrate that ELILLM can effectively enhance LLMs capabilities for SBDD.

Empowering LLMs for Structure-Based Drug Design via Exploration-Augmented Latent Inference

TL;DR

This paper tackles structure-based drug design by addressing the limitations of LLMs in understanding protein structures and guiding targeted molecule generation. It introduces ELILLM, a three-stage framework that reinterprets LLM generation as encoding, exploration in latent space via Bayesian optimization, and knowledge-guided decoding to produce chemically valid ligands. A Deep Kernel Gaussian Process surrogate with a position-aware aggregation guides latent-space exploration, while domain-informed decoding constrains outputs to plausible SMILES, resulting in higher predicted binding affinities on CrossDocked2020 and competitive performance against seven baselines. The approach demonstrates that explicit latent-space exploration can augment pretrained LLMs for domain-specific design tasks, offering interpretability, controllability, and compatibility with existing generation models. The work has practical implications for accelerating SBDD workflows and motivating broader use of uncertainty-aware latent inference in scientific AI.

Abstract

Large Language Models (LLMs) possess strong representation and reasoning capabilities, but their application to structure-based drug design (SBDD) is limited by insufficient understanding of protein structures and unpredictable molecular generation. To address these challenges, we propose Exploration-Augmented Latent Inference for LLMs (ELILLM), a framework that reinterprets the LLM generation process as an encoding, latent space exploration, and decoding workflow. ELILLM explicitly explores portions of the design problem beyond the model's current knowledge while using a decoding module to handle familiar regions, generating chemically valid and synthetically reasonable molecules. In our implementation, Bayesian optimization guides the systematic exploration of latent embeddings, and a position-aware surrogate model efficiently predicts binding affinity distributions to inform the search. Knowledge-guided decoding further reduces randomness and effectively imposes chemical validity constraints. We demonstrate ELILLM on the CrossDocked2020 benchmark, showing strong controlled exploration and high binding affinity scores compared with seven baseline methods. These results demonstrate that ELILLM can effectively enhance LLMs capabilities for SBDD.
Paper Structure (39 sections, 2 theorems, 24 equations, 8 figures, 3 tables)

This paper contains 39 sections, 2 theorems, 24 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Method Details
  4. Problem Definition
  5. Overview
  6. Molecular-Guided LLM Encoding
  7. BO-based Exploration
  8. Latent Space Sampling Strategy.
  9. Position-Aware Surrogate Model.
  10. Hierarchical Training Strategy.
  11. Acquisition Function.
  12. Knowledge-Guided LLM Decoding
  13. Experiments
  14. Experiment Setup
  15. Dataset and Settings.
  16. ...and 24 more sections

Key Result

theorem 1

Let $s = (x_1, x_2, \dots, x_l)$ be a token sequence, and let $\mathrm{tokenizer}(s)$ map $s$ to discrete token indices $(t_1, t_2, \dots, t_l)$. Let $E$ be an embedding matrix mapping each token $t$ to a vector $E[t] \in \mathbb{R}^d$, and define the token embeddings as If no positional encoding is added (e.g., no sinusoidal or learned positional embedding), then for each sequence $s$ the embedd

Figures (8)

  • Figure 1: Overview of the ELILLM framework. (A) The abstract framework of ELILLM. (B) A concrete instantiation of ELILLM, illustrating how the framework operates in a single iteration. Specifically, molecules in the observed dataset $D_\text{obs}$ are first encoded into embeddings. Next, the Latent Space Sampling Strategy samples a large set of candidate points to be evaluated. Simultaneously, the Position-Aware Surrogate Model is trained using the molecular embeddings as inputs and the corresponding docking scores as labels. The trained surrogate model is then used to predict the docking score distribution of the candidate points, and the acquisition function balances exploitation and exploration to select the most promising embeddings. Finally, through Knowledge-Guided LLM Decoding, chemical knowledge is used to constrain the LLM’s decoding behavior, converting candidate embeddings into the closest valid molecular SMILES to form a candidate molecule set. The candidate molecules are evaluated with black-box docking software, and the resulting molecule–score pairs are incorporated back into $D_\text{obs}$.
  • Figure 2: Top1 Vina docking score for different generated molecules (TargetDiff, ALIDIFF, ELILLM-rand) across 100 testing targets, sorted by Vina docking score of TargetDiff result. Trend lines are least-squares linear fits for each method.
  • Figure 3: Average Tanimoto similarity between every 10 newly generated molecules and the initial $\mathcal{D}_{\mathrm{obs}}$, including both the average of all pairwise similarities (mean similarity) and the average of per-molecule maximum similarity to $\mathcal{D}_{\mathrm{obs}}$ (max similarity). Results are averaged across 100 targets to show overall trends under two $\mathcal{D}_{\mathrm{obs}}$ construction strategies.
  • Figure 4: Visualizations of generated ligands for protein pockets 2jjg generated by ALIDIFF and ELILLM-diff.
  • Figure S1: ELILLM prompt
  • ...and 3 more figures

Theorems & Definitions (2)

  • theorem 1: Independence of Token Embeddings and Positions
  • theorem 2: Positional Unbiasedness of Aggregation