Learning Pareto-Optimal Pandemic Intervention Policies with MORL

TL;DR

The paper tackles the challenge of designing pandemic intervention policies that balance health outcomes with economic stability by employing a simulator-driven multi-objective reinforcement learning (MORL) approach. It introduces a high-fidelity, stochastic differential equation (SDE) based pandemic simulator calibrated to global data, and trains a Pareto-Conditioned Network (PCN) to learn Pareto-optimal intervention policies across diverse pathogens and outbreak severities. The framework supports policy exploration through a multi-objective reward structure and demonstrates its generality with COVID-19, polio, influenza, and measles case studies, including vaccination coverage effects. The results show how trade-offs shift with disease characteristics and urban density, offering an interpretable decision-support tool for policy makers that can adapt to future public health crises.

Abstract

The COVID-19 pandemic underscored a critical need for intervention strategies that balance disease containment with socioeconomic stability. We approach this challenge by designing a framework for modeling and evaluating disease-spread prevention strategies. Our framework leverages multi-objective reinforcement learning (MORL) - a formulation necessitated by competing objectives - combined with a new stochastic differential equation (SDE) pandemic simulator, calibrated and validated against global COVID-19 data. Our simulator reproduces national-scale pandemic dynamics with orders of magnitude higher fidelity than other models commonly used in reinforcement learning (RL) approaches to pandemic intervention. Training a Pareto-Conditioned Network (PCN) agent on this simulator, we illustrate the direct policy trade-offs between epidemiological control and economic stability for COVID-19. Furthermore, we demonstrate the framework's generality by extending it to pathogens with different epidemiological profiles, such as polio and influenza, and show how these profiles lead the agent to discover fundamentally different intervention policies. To ground our work in contemporary policymaking challenges, we apply the model to measles outbreaks, quantifying how a modest 5% drop in vaccination coverage necessitates significantly more stringent and costly interventions to curb disease spread. This work provides a robust and adaptable framework to support transparent, evidence-based policymaking for mitigating public health crises.

Figures (18)

• Figure 1: Simulated and observed new cases trajectories across Italy, USA, and UK. The simulator closely aligns with overall growth trends observed in the real data, demonstrating its ability to reproduce realistic epidemic progression across diverse geographies and population sizes. All 10 runs are scaled, with each simulated individual representing 1,000 people in reality.
• Figure 2: Number of new infections, deaths, and quarantined individuals, with corresponding interventions, throughout a 50-day episode with 1,000 initial infections ($\approx1.5\%$ of the population) when the agent prioritizes (left) balancing disease spread and economic disruption, (middle) mitigating infections, and (right) economic welfare. On the left, the agent applies moderate, sustained measures to control spread without severely restricting economic activity. In the middle, stringent, prolonged interventions curb the spread. On the right, no interventions are applied, demonstrating an extreme prioritization of economic activity, resulting in a sharp increase in new infections.
• Figure 3: Pareto fronts with 1,000 initial infections when the agent prioritizes (left) balancing disease spread and economic disruption, (middle) mitigating infections, and (right) economic welfare. Corresponding to \ref{['fig:comparison_trajectories']}, the approximated Pareto fronts show the optimal trade-offs between new infections, new deaths, and economic costs when prioritizing each intervention strategy.
• Figure 4: Comparison of intervention strategies and outcomes under higher daily contact rates of $\mu=15$ and $\mu=20$. As contact rate increases, interventions must become substantially more stringent and prolonged to contain disease spread (see Appendix \ref{['appendix:interventions_contact_rates']}), leading to disproportionately higher economic burdens.
• Figure 5: Comparison of intervention strategies for polio and influenza under the same initial conditions and objectives as the COVID-19 baseline. Polio requires far stronger interventions (see Appendix \ref{['appendix:interventions_polio_influenza']}) and still incurs higher health losses, while influenza can be managed with mild measures and minimal economic disruption.