Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

TL;DR

A unified framework that addresses both the reaction-representation learning and molecule generation tasks, which allows for a more holistic approach and develops a new pretraining framework that enables us to incorporate inductive biases into the model.

Abstract

Chemical reactions are the fundamental building blocks of drug design and organic chemistry research. In recent years, there has been a growing need for a large-scale deep-learning framework that can efficiently capture the basic rules of chemical reactions. In this paper, we have proposed a unified framework that addresses both the reaction representation learning and molecule generation tasks, which allows for a more holistic approach. Inspired by the organic chemistry mechanism, we develop a novel pretraining framework that enables us to incorporate inductive biases into the model. Our framework achieves state-of-the-art results on challenging downstream tasks. By possessing chemical knowledge, our generative framework overcome the limitations of current molecule generation models that rely on a small number of reaction templates. In the extensive experiments, our model generates synthesizable drug-like structures of high quality. Overall, our work presents a significant step toward a large-scale deep-learning framework for a variety of reaction-based applications.

Figures (10)

• Figure 1: An overview of the unified framework of Uni-RXN. The atom mapping is utilized for splitting the reactants into the main reactant and sub-reactants. Then, a multimodal graph/text-based transformer model is applied for deriving the chemical entity-level representations. Entity-level representations are further fused into reaction-level representations.
• Figure 2: (a) Two contrastive learning tasks we utilized for pretraining. The main reactant, sub-reactants, and reagents are projected into the embedding space using different attention-based model architectures. The similarity between the embeddings of {main reactant} and {sub reactants, reagents} is maximized because of the underlying organic mechanism. The similarity between the embeddings of {main reactant, sub reactants, reagents} and {product} is maximized because of the graph-level mapping. (b) An illustration of the reactive center prediction task. (c) The model architecture for contrastive learning. $c$ stands for the chemical training signal which is applied in the first contrastive learning objective. While $g$ stands for the graph training signal which is applied in the second contrastive learning objective.. (d) The model architecture for reactive center prediction tasks. An additional graph-based transformer model is applied to identify the place where chemical bonds are broken or newly formed in chemical reactions.
• Figure 3: (a)The line charts illustrate the AUROC, EF$_1$$_{\%}$ and EF$_1$$_{\%}$ of different representations in the chemical reaction retrieval task. (b)The attention map of the graph-based transformer encoder. Atoms that belong to the same functional group have high cross-attention which demonstrates that our model is capable of learning position effect and identifying reactive atoms.
• Figure 4: (a) An overview of the generation framework. A sequential process is proposed to generate the analogues. At each step, the model proposed the sub-reactants and reagents, then the reaction predictor outputs the product. The product is fed to the model as the input for the next step until the termination criterion is met.(b) The model architecture of the generative model. The pre-trained model is utilized here as the encoder for target structures and main reactant inputs.
• Figure 5: The properties of the drug-like molecule generated by different reaction-based molecular generation models. The MMD distribution distances are listed within the ().