BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

TL;DR

This work addresses the challenge of designing novel active molecules within protein binding pockets by introducing BindGPT, a decoder-only language-model framework that generates 3D molecular graphs and conformations as text tokens (SMILES and XYZ). Through a scalable pretraining-finetuning-RL pipeline, BindGPT pretrains on a large 3D molecular dataset, finetunes on CrossDocked for pocket-ligand pairs, and further refines with docking-based reinforcement learning, achieving competitive or superior results to diffusion and GNN baselines while offering substantially faster sampling. The approach demonstrates strong performance on unconditional 3D molecule generation, zero-shot conformer generation, and notably superior pocket-conditioned generation, with RL yielding the best binding affinities and robust drug-likeness metrics. Overall, BindGPT showcases a data-efficient, generalizable pathway to leverage NLP-style pretraining for 3D drug design, enabling scalable, prompt-driven generation in realistic protein environments.

Abstract

Generating novel active molecules for a given protein is an extremely challenging task for generative models that requires an understanding of the complex physical interactions between the molecule and its environment. In this paper, we present a novel generative model, BindGPT which uses a conceptually simple but powerful approach to create 3D molecules within the protein's binding site. Our model produces molecular graphs and conformations jointly, eliminating the need for an extra graph reconstruction step. We pretrain BindGPT on a large-scale dataset and fine-tune it with reinforcement learning using scores from external simulation software. We demonstrate how a single pretrained language model can serve at the same time as a 3D molecular generative model, conformer generator conditioned on the molecular graph, and a pocket-conditioned 3D molecule generator. Notably, the model does not make any representational equivariance assumptions about the domain of generation. We show how such simple conceptual approach combined with pretraining and scaling can perform on par or better than the current best specialized diffusion models, language models, and graph neural networks while being two orders of magnitude cheaper to sample.

Figures (15)

• Figure 1: The pipeline of our 3D molecule pretraining-finetuning paradigm. During pretraining the model is trained on a mix of molecules and pockets in isolation. But at finetuning, we simply concatenate the pocket text representation and the molecule text representation for each pocket-ligand pair.
• Figure 2: Data layout during the pretraining. Arrows show the tokens sequence order. Nodes such as <POCKET> show special tokens. Training is done on a mixture of pocket and ligand datasets. Concrete examples are shown in Appendix \ref{['appe:tokenization']}.
• Figure 3: Data layout during the finetuning. Arrows show the tokens sequence order. Nodes such as <POCKET> show special tokens. Concrete examples are shown in Appendix \ref{['appe:tokenization']}.
• Figure 4: (left) Sampled conformations for reference molecules from the Platinum dataset (See Appendix \ref{['app:more_samples']} for a higher resolution image) (right) RMSD Coverage metric calculated on the Platinum dataset for the 3D conformation generation task.
• Figure 5: Examples of binding poses and Vina scores ($\downarrow$) for 2gns and 4d7o pockets.