Semi-supervised Latent Bayesian Optimization for Designing Antimicrobial Peptides

TL;DR

This work tackles the challenge of discovering potent antimicrobial peptides by combining latent-space optimization with semi-supervised learning and dimensionality reduction. It uses a Transformer-based TransVAE to learn peptide representations, jointly trained with predictors for physicochemical properties, and evaluates Bayesian optimization in both the full latent space and PCA-reduced projections. The key finding is that latent spaces organized by multiple properties remain interpretable even with sparse labels, and that BO in PCA-projected spaces can outperform BO in the full latent space, especially in low-label settings, due to increased exploration and improved search efficiency. These results support a practical strategy for AMP design: leverage semi-supervised latent-space organization and PCA projections to enable efficient, interpretable optimization when experimental data are scarce and expensive.

Abstract

Antimicrobial peptides (AMPs) are a promising class of therapeutics to treat bacterial infections. Discovering and designing such peptides is difficult because of the vast number of possible sequences of amino acids. Deep generative models, such as variational autoencoders, have shown value in peptide design due to their ability to model sequence space with a continuous-valued latent space. Although such models have already been used to great effect in biomolecular design, they still suffer from a lack of interpretability and rigorous quantification of latent space quality as a search space. We investigate (1) whether further compression of the design space via dimensionality reduction may facilitate optimization, (2) the interpretability of the spaces, and (3) how organizing latent spaces with physicochemical properties may improve the efficiency of optimizing antimicrobial activity. We find that further reduction of the latent space via dimensionality reduction can be advantageous when organizing the space with more relevant information at data availability, that using the dimensionality reduction search space can be more interpretable, and that we can organize the latent space with different physicochemical properties even at different percentages of available labels.

Figures (28)

• Figure 1: PCA-projected transVAE latent spaces jointly-trained by Boman index, hydrophobicity, and charge, at varying percentages of property values. The left column is coloured by Boman index. The middle column is coloured by charge. The right column is coloured by hydrophobicity. Barplots share x-axes with its corresponding scatterplot; bar heights are the average value of the column's property averaged over the points lying in a bar's x-axis interval width; monotonic trends suggest the (average in) property changes monotonically along the shared x-axis. Both the scatterplot and its shared barplot have the same colourmap. The shown principal components are the two PCs most correlated with the given property value, as measured by a Pearson r value (see SI Table \ref{['tbl:si:PCs_in5x3']} for each PC pair). We observe property organization at each percentage of masking, and for each property, although with some switching in direction.
• Figure 2: Quantifying the distortion induced by using PCA on the latent spaces. For each quadrant, from left to right we have the property (or properties) used to organize the latent space; starting on the left, Boman index, charge (pH=7), hydrophobicity, Boman and charge (pH=7), and all three. Variability in each quantity is estimated with 35 equally-sized subsamples of 42,000 different points in the latent space. (a) The estimated trustworthiness of the PCA space compared to the original 64-dimensional latent space. (b) The estimated continuity of the PCA space compared to the latent space. (c) The estimated steadiness of the PCA space compared to the latent space. (d) The estimated cohesiveness of the PCA space compared to the latent space.
• Figure 3: Difference in Bayesian optimization results $S_{\text{diff}}$, as defined in Eqn. \ref{['eqn:s-diff']}. Error bars correspond to the standard error of the propagated uncertainty. Note that the the x-axis range is only up to iteration 100. We observe that searching through the PCA'ed latent space offers an advantage in earlier iterations compared to searching through the 64-dimensional latent space. When we have access to only 2% of the labels to organize the VAE's latent space, 20 PCA components performs best.
• Figure 4: Best scores identified through Bayesian optimization runs while varying the search space dimensionality. Bayesian optimization results for searching through the 64-dimensional latent space (blue line, "Id" for "Identity projection"), and for searching through the projected latent space with varying numbers of PCA components, of BCH organized VAE (a,b) and of oracle-organized VAE (c,d). Label percentage corresponds to the percentage of property labels available during VAE training. Objective scores $\langle \mathcal{M}\rangle$ averaged over five BayesOpt runs with randomly initialized starting points. Error bars represent standard errors across the five runs.
• Figure 5: Average best objective scores found at different iterations. Error bars correspond to the standard error of $M_{\text{best}}$ over five BayesOpt runs. Gray dashed line separates those runs performed in the high-dimensional latent space (above the dashed line), and those runs performed in a PCA projection of the latent space (below the dashed line). The full BayesOpt trajectories are depicted in SI Fig. \ref{['fig:boloops:exploitation:foreach-perc-vary-prop']}.