A Multi-Modal Contrastive Diffusion Model for Therapeutic Peptide Generation

TL;DR

This work tackles the challenge of generating therapeutic peptides by jointly modeling sequence and 3D structure. It introduces MMCD, a multi-modal diffusion framework that co-generates peptide sequences and backbone coordinates, guided by inter- and intra-modal contrastive learning to align modalities and discriminate therapeutic versus non-therapeutic peptides. Empirical results on AMP and ACP datasets show MMCD outperforms baselines in antimicrobial/anticancer scores, structural validity, docking performance, and diversity, with ablations confirming the importance of both Inter-CL and Intra-CL. The approach demonstrates a significant advance in peptide design by leveraging cross-modal information and contrastive signals to improve generalization and functional quality.

Abstract

Therapeutic peptides represent a unique class of pharmaceutical agents crucial for the treatment of human diseases. Recently, deep generative models have exhibited remarkable potential for generating therapeutic peptides, but they only utilize sequence or structure information alone, which hinders the performance in generation. In this study, we propose a Multi-Modal Contrastive Diffusion model (MMCD), fusing both sequence and structure modalities in a diffusion framework to co-generate novel peptide sequences and structures. Specifically, MMCD constructs the sequence-modal and structure-modal diffusion models, respectively, and devises a multi-modal contrastive learning strategy with intercontrastive and intra-contrastive in each diffusion timestep, aiming to capture the consistency between two modalities and boost model performance. The inter-contrastive aligns sequences and structures of peptides by maximizing the agreement of their embeddings, while the intra-contrastive differentiates therapeutic and non-therapeutic peptides by maximizing the disagreement of their sequence/structure embeddings simultaneously. The extensive experiments demonstrate that MMCD performs better than other state-of-theart deep generative methods in generating therapeutic peptides across various metrics, including antimicrobial/anticancer score, diversity, and peptide-docking.

Figures (4)

• Figure 1: Overview of the MMCD. MMCD consists of a diffusion model for the peptide sequence-structure co-generation and multi-modal contrastive learning (CL). The diffusion model involves a forward process ($q(\cdot|\cdot)$) for adding noise and a reverse process ($p(\cdot|\cdot)$) for denoising at each timestep $t$. The reverse process utilizes a transformer encoder (or EGNN) to extract embeddings from sequences $S$ (or structures $C$), and a sequence (or structure)-based MLP to map embeddings to the marginal distribution (or Gaussian) noise. The multi-modal CL includes an Inter-CL and an Intra-CL, which aims to align sequence and structure embeddings, and differentiate therapeutic and non-therapeutic peptide embeddings.
• Figure 2: (a) The sample ratio under different sequence lengths in the AMP dataset, where the red line is the average ratio. (b) The similarity and RMSD scores of MMCD and baselines across different sequence lengths.
• Figure 3: (a) The t-SNE for structure and sequence embeddings of therapeutic peptides (AMP data) obtained from MMCD (w/o Inter-CL) and MMCD. (b) The t-SNE for embeddings (including structures and sequences) of therapeutic (AMP) and non-therapeutic (non-AMP) peptides obtained from MMCD (w/o Intra-CL) and MMCD.
• Figure 4: Docking analysis (interactive visualization between target protein and peptides) of the reference and generated structures by MMCD and baselines. Thick lines represent the residues of peptides, and the thin lines show the binding residues for protein-peptide complexes.