Biomanufacturing Harvest Optimization with Small Data
Bo Wang, Wei Xie, Tugce Martagan, Alp Akcay, Bram van Ravenstein
TL;DR
This work tackles optimal fermentation harvesting under severe data scarcity by marrying mechanistic protein/impurity growth with Bayesian learning and Markov decision processes. It develops a model with $p_{t+1}=p_t e^{\Phi_t}$ and $i_{t+1}=i_t e^{\Psi_t}$, where $\Phi_t \sim \mathcal{N}(\mu^{(p)}_c,\sigma^{(p)2}_c)$ and $\Psi_t \sim \mathcal{N}(\mu^{(i)}_c,\sigma^{(i)2}_c)$, and updates unknown parameters via a Normal–Inverse–Gamma prior to obtain posterior predictive distributions $\widetilde{\Phi}_t$ and $\widetilde{\Psi}_t$ with generalized $t$-distributions. The harvesting problem is formulated as an MDP with a knowledge state $\mathcal{I}_t$ capturing posterior parameters and a hyper-state $\mathcal{H}_t=(p_t,i_t,\mathcal{I}_t)$, featuring a control-limit structure on impurity and a myopic policy that can be optimal under perfect information when certain conditions hold. The paper develops RL with model risk (RL with MR) using Bayesian sparse sampling and Thompson sampling to compute near-optimal policies online, and demonstrates a real MSD implementation that yields about a 50% increase in batch yield with reduced variability. Overall, accounting for model risk and leveraging small-data Bayesian learning significantly improves fermentation harvesting decisions and operational performance, with clear pathways for extension to continuous-time settings and broader biomanufacturing contexts. The results underscore the value of data-driven decision-making in early-stage bioprocess development and the practical impact of integrate learning and optimization under uncertainty.
Abstract
In biopharmaceutical manufacturing, fermentation processes play a critical role in productivity and profit. A fermentation process uses living cells with complex biological mechanisms, leading to high variability in the process outputs, namely, the protein and impurity levels. By building on the biological mechanisms of protein and impurity growth, we introduce a stochastic model to characterize the accumulation of the protein and impurity levels in the fermentation process. However, a common challenge in the industry is the availability of only a very limited amount of data, especially in the development and early stages of production. This adds an additional layer of uncertainty, referred to as model risk, due to the difficulty of estimating the model parameters with limited data. In this paper, we study the harvesting decision for a fermentation process (i.e., when to stop the fermentation and collect the production reward) under model risk. We adopt a Bayesian approach to update the unknown parameters of the growth-rate distributions, and use the resulting posterior distributions to characterize the impact of model risk on fermentation output variability. The harvesting problem is formulated as a Markov decision process model with knowledge states that summarize the posterior distributions and hence incorporate the model risk in decision-making. Our case studies at MSD Animal Health demonstrate that the proposed model and solution approach improve the harvesting decisions in real life by achieving substantially higher average output from a fermentation batch along with lower batch-to-batch variability.