3D-Mol: A Novel Contrastive Learning Framework for Molecular Property Prediction with 3D Information

TL;DR

3D-Mol introduces a hierarchical 3D molecular encoder that decouples atom-bond, bond-angle, and dihedral-angle information into three graphs $G_{a-b}$, $G_{b-a}$, and $G_{d-a}$. It couples this encoder with a weighted contrastive pretraining scheme over 20M unlabeled conformations, where conformations sharing the same SMILES are weighted positives and others are negatives weighted by 3D descriptor and fingerprint similarities, and augments this with geometry-centric pretraining tasks. The model, pretrained on large unlabeled data and finetuned on MoleculeNet benchmarks, achieves state-of-the-art or near-state-of-the-art performance on multiple datasets, notably excelling on BACE and several regression tasks, and outperforms non-pretrained and many pretrained baselines. Ablation studies confirm the contributions of the dihedral-angle graph, the weighted contrastive weighting, and the overall pretraining strategy. While effective, the approach hinges on the computationally intensive generation of 3D conformations, pointing to future work on efficiency enhancements to broaden practical applicability in drug discovery pipelines.

Abstract

Molecular property prediction, crucial for early drug candidate screening and optimization, has seen advancements with deep learning-based methods. While deep learning-based methods have advanced considerably, they often fall short in fully leveraging 3D spatial information. Specifically, current molecular encoding techniques tend to inadequately extract spatial information, leading to ambiguous representations where a single one might represent multiple distinct molecules. Moreover, existing molecular modeling methods focus predominantly on the most stable 3D conformations, neglecting other viable conformations present in reality. To address these issues, we propose 3D-Mol, a novel approach designed for more accurate spatial structure representation. It deconstructs molecules into three hierarchical graphs to better extract geometric information. Additionally, 3D-Mol leverages contrastive learning for pretraining on 20 million unlabeled data, treating their conformations with identical topological structures as weighted positive pairs and contrasting ones as negatives, based on the similarity of their 3D conformation descriptors and fingerprints. We compare 3D-Mol with various state-of-the-art baselines on 7 benchmarks and demonstrate our outstanding performance.

Figures (6)

• Figure 1: Geometric difference leads to diverse properties. Thalidomide exists in two distinct 3D stereoisomeric forms, known as R-Thalidomide and S-Thalidomide. These two molecules can be represented by the same SMILES, but they have significantly dissimilar properties. The former is recognized for its therapeutic properties, while the latter has been implicated in teratogenesis.
• Figure 2: The overview of the 3D-Mol model framework. a) In the pretraining stage, we employ weighted contrastive learning to effectively pretrain our model. In addition to using the mask strategy for graph data augmentation, we consider conformations from the same SMILES as positive pairs, while the weight represents their 3D conformation descriptor similarity. Conversely, distinct topological structures are treated as negative pairs, and the weight is dependent on fingerprint differences. b) In the finetuning stage, we use one well-pretrained encoder model to refine our approach across diverse downstream datasets through supervised learning.
• Figure 3: The overview of the 3D-Mol encoder layer. The 3D-Mol encoder layer comprises three steps. Firstly, employing a message passing strategy, nodes in each graph exchange messages with their connected edges, leading to the updating of edge and node latent vectors. Secondly, the edge latent vector from the lower-level graph is transmitted to the higher-level graph as part of the node latent vector. Finally, the iteration is performed n times to derive the $n_{th}$ node latent vector, from which we extract the molecular latent vectors.
• Figure 4: The overview of weighted positive/negative pairs. Using the molecule Oc1ccc(cc1)CC[NH3+]. a). Low weight positive pairs: Features two conformations with the same SMILES and significant structural differences (e.g., chirality, geometric angles). b). High weight positive pairs: Depicts conformations with the same SMILES but minor differences, such as slight dihedral angle variations. c). Low weight negative pair: Shows conformations from different SMILES with similar scaffolds but missing little functional group, like an (-OH) group. d). High weight negative pair: Illustrates conformations from different SMILES, differing greatly in both scaffold and functional groups.
• Figure 5: Case study of 3D information enhance using 3D-Mol for the BACE task. This figure illustrates the prediction results for three molecules identified as BACE inhibitors, showcasing the efficacy of 3D-Mol, which utilizes three-dimensional molecular information, versus GIN, which relies on two-dimensional data. Each molecule is displayed with its respective name (Q27467123, SCHEMBL12917066, Q27455563), and the results indicate that 3D-Mol accurately predicts BACE inhibition across all cases, while GIN fails to do so.