Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GROOT: Effective Design of Biological Sequences with Limited Experimental Data

Thanh V. T. Tran, Nhat Khang Ngo, Viet Anh Nguyen, Truong Son Hy

TL;DR

GROOT is introduced, a GRaph-based Latent SmOOThing for Biological Sequence Optimization that generates pseudo-labels for neighbors sampled around the training latent embeddings, demonstrating its practicality and effectiveness.

Abstract

Latent space optimization (LSO) is a powerful method for designing discrete, high-dimensional biological sequences that maximize expensive black-box functions, such as wet lab experiments. This is accomplished by learning a latent space from available data and using a surrogate model to guide optimization algorithms toward optimal outputs. However, existing methods struggle when labeled data is limited, as training the surrogate model with few labeled data points can lead to subpar outputs, offering no advantage over the training data itself. We address this challenge by introducing GROOT, a Graph-based Latent Smoothing for Biological Sequence Optimization. In particular, GROOT generates pseudo-labels for neighbors sampled around the training latent embeddings. These pseudo-labels are then refined and smoothed by Label Propagation. Additionally, we theoretically and empirically justify our approach, demonstrate GROOT's ability to extrapolate to regions beyond the training set while maintaining reliability within an upper bound of their expected distances from the training regions. We evaluate GROOT on various biological sequence design tasks, including protein optimization (GFP and AAV) and three tasks with exact oracles from Design-Bench. The results demonstrate that GROOT equalizes and surpasses existing methods without requiring access to black-box oracles or vast amounts of labeled data, highlighting its practicality and effectiveness. We release our code at https://anonymous.4open.science/r/GROOT-D554

GROOT: Effective Design of Biological Sequences with Limited Experimental Data

TL;DR

GROOT is introduced, a GRaph-based Latent SmOOThing for Biological Sequence Optimization that generates pseudo-labels for neighbors sampled around the training latent embeddings, demonstrating its practicality and effectiveness.

Abstract

Latent space optimization (LSO) is a powerful method for designing discrete, high-dimensional biological sequences that maximize expensive black-box functions, such as wet lab experiments. This is accomplished by learning a latent space from available data and using a surrogate model to guide optimization algorithms toward optimal outputs. However, existing methods struggle when labeled data is limited, as training the surrogate model with few labeled data points can lead to subpar outputs, offering no advantage over the training data itself. We address this challenge by introducing GROOT, a Graph-based Latent Smoothing for Biological Sequence Optimization. In particular, GROOT generates pseudo-labels for neighbors sampled around the training latent embeddings. These pseudo-labels are then refined and smoothed by Label Propagation. Additionally, we theoretically and empirically justify our approach, demonstrate GROOT's ability to extrapolate to regions beyond the training set while maintaining reliability within an upper bound of their expected distances from the training regions. We evaluate GROOT on various biological sequence design tasks, including protein optimization (GFP and AAV) and three tasks with exact oracles from Design-Bench. The results demonstrate that GROOT equalizes and surpasses existing methods without requiring access to black-box oracles or vast amounts of labeled data, highlighting its practicality and effectiveness. We release our code at https://anonymous.4open.science/r/GROOT-D554

Paper Structure

This paper contains 44 sections, 2 theorems, 8 equations, 3 figures, 9 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related work
  4. LSO for Sequence Design
  5. Protein Fitness Regularization
  6. Method
  7. Problem Formulation
  8. Constructing Latent Space of Protein Sequences
  9. VAE's Architecture
  10. Encoder
  11. Decoder
  12. Latent Graph-based Smoothing
  13. Graph Construction
  14. Generating Pseudo-label
  15. Theoretical Justification
  16. ...and 29 more sections

Key Result

Proposition 1

Let $\mathbb{X} = \{x_i \in \mathbb{R}^d\}_{i=1}^N$ be a set of $N$ i.i.d $d$-dimensional samples from $\mathcal{N}(0,I_d)$. Assume that $N \ll \exp (\frac{d}{2(C^2_{\beta}+2)})$ and let $z = \beta * \overline x + (1-\beta) * \epsilon$ for some $\overline x \in \mathbb{X}$, $\beta \in (0,1)$, $C_{\b

Figures (3)

  • Figure 1: Overall framework of GROOT. After encoding sequences into the latent space, we generate new samples by adding Gaussian noise to existing vectors. These synthetic data lie outside the training set's convex hull but within a reliable zone, as their distances from the hull are below a certain upper bound. We construct a kNN graph and run label propagation to smooth and refine node labels. These nodes and their fitness values are then used to train the surrogate model, which is subsequently employed for optimization.
  • Figure 2: Distance from generated nodes outside the convex hull to the set of training nodes. The mean and standard deviation over 5 different runs are reported. We set $\boldsymbol{\beta=0.5}$ to maintain a constant upper bound on the distance.
  • Figure 3: The performance of GROOT on AAV and GFP harder3 tasks when we vary the labeled data ratio $r$. The mean and standard deviation over 5 different runs are reported.

Theorems & Definitions (4)

  • definition 1
  • definition 2: extrapolate
  • Proposition 1
  • Proposition 2