ProDCARL: Reinforcement Learning-Aligned Diffusion Models for De Novo Antimicrobial Peptide Design

TL;DR

ProDCARL tackles antimicrobial peptide design under rising resistance by integrating a diffusion-based generator with reinforcement-learning alignment to optimize for high AMP activity and low toxicity. A domain-adapted EvoDiff prior is fine-tuned on AMP sequences and guided by two predictor networks for activity and toxicity, using a top-$k$ policy-gradient objective with entropy regularization to preserve diversity. In silico results show mean AMP score rising from $0.081$ to $0.178$, a joint high-quality hit rate of $6.3\%$, and diversity $1 - \text{mean pairwise identity} = 0.929$, with AlphaFold3 and ProtBERT analyses supporting plausible AMP-like candidates. The framework provides a computational triage tool to narrow experimental search space, though wet-lab validation and stronger safeguards against reward hacking are needed for practical translation.

Abstract

Antimicrobial resistance threatens healthcare sustainability and motivates low-cost computational discovery of antimicrobial peptides (AMPs). De novo peptide generation must optimize antimicrobial activity and safety through low predicted toxicity, but likelihood-trained generators do not enforce these goals explicitly. We introduce ProDCARL, a reinforcement-learning alignment framework that couples a diffusion-based protein generator (EvoDiff OA-DM 38M) with sequence property predictors for AMP activity and peptide toxicity. We fine-tune the diffusion prior on AMP sequences to obtain a domain-aware generator. Top-k policy-gradient updates use classifier-derived rewards plus entropy regularization and early stopping to preserve diversity and reduce reward hacking. In silico experiments show ProDCARL increases the mean predicted AMP score from 0.081 after fine-tuning to 0.178. The joint high-quality hit rate reaches 6.3\% with pAMP $>$0.7 and pTox $<$0.3. ProDCARL maintains high diversity, with $1-$mean pairwise identity equal to 0.929. Qualitative analyses with AlphaFold3 and ProtBERT embeddings suggest candidates show plausible AMP-like structural and semantic characteristics. ProDCARL serves as a candidate generator that narrows experimental search space, and experimental validation remains future work.

Figures (5)

• Figure 1: Overview of the ProDCARL pipeline. The framework integrates data preparation, classifier training, EvoDiff prior fine-tuning, and reinforcement learning alignment with rewards.
• Figure 2: Quantitative analysis of diffusion prior fine-tuning. (a) Training trajectory for negative log-likelihood loss and a 50-batch moving average with $\pm 1\sigma$ shading. (b) Distribution of loss values for the final 100 batches. (c) Late-stage loss values for steps 650-800.
• Figure 3: Qualitative structural comparison of natural and ProDCARL-generated peptides (AlphaFold3). Several generated candidates show predominantly $\alpha$-helical conformations consistent with common AMP motifs. Predictions are used as qualitative plausibility checks for short peptides.
• Figure 4: ProtBERT-UMAP visualization comparing ProDCARL-generated peptides to natural AMPs (APD). The overlap suggests alignment in representation space, while the narrower spread of generated candidates likely reflects fixed length and reward shaping rather than the full natural AMP diversity.
• Figure 5: Reward and diversity during reinforcement learning. "EvoDiff+RL" denotes RL updates from the base EvoDiff model without AMP fine-tuning. "EvoDiff+FT" denotes the AMP fine-tuned prior without RL updates. ProDCARL initializes RL from the AMP fine-tuned prior and applies top-$k$ updates with entropy regularization and early stopping.