Simulation-Based Optimization over Discrete Spaces using Projection to Continuous Latent Spaces

TL;DR

Addresses the challenge of optimizing complex, discrete design spaces in black-box simulations. Proposes a framework that combines a Variational AutoEncoder (VAE) to map discrete designs to a continuous latent space $\mathbf{z}\in\mathcal{Z}$ and Bayesian Optimization (BO) with a Gaussian Process surrogate to minimize $f(\mathbf{x})$ via $\min_{\mathbf{z}\in\mathcal{Z}} f(\phi^{-1}(\mathbf{z}))$. Demonstrates two AspenPlus-based case studies—caprylic acid extraction and a dividing-wall column—where BO+VAE rapidly identifies Pareto-optimal designs with a small number of simulations. Highlights structured latent-space regions corresponding to different solvents and configurations, enabling efficient exploration under limited budgets.

Abstract

Simulation-based optimization of complex systems over discrete decision spaces is a challenging computational problem. Specifically, discrete decision spaces lead to a combinatorial explosion of possible alternatives, making it computationally prohibitive to perform simulations for all possible combinations. In this work, we present a new approach to handle these issues by transforming/projecting the discrete decision space into a continuous latent space using a probabilistic model know as Variational AutoEncoders. The transformation of the decision space facilitates the implementation of Bayesian optimization (BO), which is an efficient approach that strategically navigates the space to reduce the number of expensive simulations. Here, the key observation is that points in the latent space correspond to decisions in the original mixed-discrete space, but the latent space is much easier to navigate using BO. We illustrate the benefits of our approach through a couple of case studies that aim to design complex distillation systems: the recovery of caprylic acid from water by liquid-liquid extraction and the separation of an azeotropic mixture using a thermally couple column know as extractive dividing wall column.

Figures (10)

• Figure 1: VAE architecture designed for a set of 3 discrete inputs $\mathbf{x} = [x_1, x_2, x_3]\in \mathcal{X}$; each discrete input is mapped into a continuous representation $\mathbf{e}$ through an embedding layer. The embeddings are concatenated into a vector $\textbf{h}$ and processed by the encoder. The encoder produces the mean vector $\boldsymbol{\mu}$ and the standard deviation vector $\boldsymbol{\sigma}$, which are used to generate the latent space $\mathcal{Z}$. The decoder maps points in the latent space $\mathbf{z}\in \mathcal{Z}$ back into the original space by producing logits $\mathbf{y}$ for each input variable; a softmax function is used to obtain probability distributions over the possible values of $x_1, x_2$, and $x_3$ to reconstruct the original discrete space (analogous to a classification model).
• Figure 2: Schematic representation of the original discrete space $\mathcal{X}$ mapped through the decoder $\phi$ to obtain the latent continuous space $\mathcal{Z}$ from which we can reconstruct the original space using the decoder $\phi^{-1}$.
• Figure 3: Impact of the latent space dimension on the composite loss function $\mathcal{L}$ and its components: the reconstruction $\mathcal{L}_{\mathrm{rec}}$ and the KL-divergence $\mathcal{L}_{\mathrm{KL-D}}$ functions (left). Progression of loss functions during the training of a VAE with a latent dimension of eight.
• Figure 4: Illustration of integrated BO+VAE framework. Data $\textbf{x}$ is mapped to $\textbf{z}$ using $\phi$. A surrogate model over the latent space $m(\mathbf{z})$ is built and iteratively selects promising points in the latent space $\mathbf{z}^{{\ell+1}}$ by optimizing an acquisition function ($AF$). Latent samples $\mathbf{z}^{\ell+1}$ are decoded to obtain a simulation at the new point $f(\phi^{-1}(\mathbf{z}^{\ell+1}))$. The surrogate model is refined by updating the dataset $\mathcal{D}^{\ell}$.
• Figure 5: Process for the recovery of caprylic acid using liquid-liquid extraction.