Time-Aware Latent Space Bayesian Optimization

TL;DR

Time-Aware Latent-space Latent-space Bayesian Optimization (TALBO) is proposed, which incorporates time in both the surrogate and the learned generative representation via a GP-prior variational autoencoder, yielding a latent space aligned as objectives evolve.

Abstract

Latent-space Bayesian optimization (LSBO) extends Bayesian optimization to structured domains, such as molecular design, by searching in the continuous latent space of a generative model. However, most LSBO methods assume a fixed objective, whereas real design campaigns often face temporal drift (e.g., evolving preferences or shifting targets). Bringing time-varying BO into LSBO is nontrivial: drift can affect not only the surrogate, but also the latent search space geometry induced by the representation. We propose Time-Aware Latent-space Bayesian Optimization (TALBO), which incorporates time in both the surrogate and the learned generative representation via a GP-prior variational autoencoder, yielding a latent space aligned as objectives evolve. To evaluate timevarying LSBO systematically, we adapt widely used molecular design tasks to drifting multi-property objectives and introduce metrics tailored to changing targets. Across these benchmarks, TALBO consistently outperforms strong LSBO baselines and remains robust across drift speeds and design choices, while remaining competitive under actually time-invariant objectives.

Figures (9)

• Figure 1: Overview of our setting. At each iteration $t$, a black-box objective $f_t$ is defined as a weighted combination of two molecular properties, with a predefined time-varying weight schedule (details in Section \ref{['sec:mpo']}). Left: The best score under the current objective $f_t$ among all molecules evaluated up to iteration $t$, (mean $\pm$ std over 10 seeds with different initial datasets). Top-right: mean property components. Bottom-right: objective weights (identical across seeds). TALBO, our time-aware method adapts more rapidly to changes in objective importance, resulting in consistently higher attained values $\tilde{f}_t$.
• Figure 2: Performance comparison across dynamic multi-objective tasks. First row: current-objective best-so-far value. Second row: cumulative regret, expressed as percentage improvement relative to InvBO. Third row: baseline rank at each iteration based on cumulative regret (lower is better). Fourth row: time-varying objective weights governing the dynamic multi-objective trade-off, using $\ell_w = 0.2$ for $k_w$ (Equation \ref{['eq:weight']}). Mean $\pm \tfrac{1}{2}$std over 10 random seeds. All trajectories are smoothed using a moving average for improved readability. Across all tasks, TALBO consistently outperforms other baselines, attaining higher best-found values, lower regret, and better average rank.
• Figure S1: Performance comparison with pre-training dataset baselines. Current-objective best-so-far value for each dynamic multi-objective task, augmented with statistics derived from the pre-training dataset. Colored curves correspond to optimization baselines (mean $\pm \tfrac{1}{2}$ std over 10 random seeds). The solid black curve denotes the best achievable value within the pre-training dataset at each time step, i.e., $\max_j f_t(x_j)$ evaluated over all molecules in the dataset. Dashed black curves indicate top-percentile subsets of the dataset (5%, 1%, 0.1%, and 0.01% best), providing reference performance levels attainable without adaptive optimization. All trajectories are smoothed using a centered moving average for improved readability. The comparison highlights both how challenging it is to outperform strong pre-training set reference levels with only a few thousand queries, and the extent to which dynamic BO methods can nonetheless surpass best-in-dataset metrics.
• Figure S2: Per-component dataset optima under drifting priorities. For each task, we evaluate each objective component on the full GuacaMol pre-training corpus (1.27M molecules) using the same time-dependent weighting schedule as in the benchmark, and plot the best attainable component value at each iteration. The resulting trajectories exhibit clear regime changes and transitions across components, confirming that the weight schedule induces meaningful time variation in the underlying objective landscape.
• Figure S3: Ablation on the dual temporal modeling mechanism. For each task and iteration, baselines are ranked within each seed using cumulative regret, and then averaged across 10 random seeds. Curves show the mean $\pm \tfrac{1}{2}$ standard deviation over seeds. The setting is similar as Figure \ref{['fig:results']}: $\ell_w=0.2$. Explicitly modeling time leads to overall performance gains, though the relative contribution of temporal modeling in the generative model versus the surrogate varies across tasks.