Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical vs. Deep Learning Models for Estimating Substance Overdose Excess Mortality in the US

Sukanya Krishna, Marie-Laure Charpignon, Maimuna Majumder

TL;DR

This study tackles excess mortality estimation from US substance overdoses during the COVID-19 era by benchmarking traditional SARIMA against deep learning architectures (LSTM, Seq2Seq, Transformer) for counterfactual projection using CDC WONDER data. Through pre-pandemic validation and conformal prediction for uncertainty, it finds that LSTM provides the best balance of point accuracy and calibrated prediction intervals, while attention-based models overfit to historical means and SARIMA undercaptures nonlinear regime changes. The work delivers a reproducible open-source pipeline and demonstrates substantial cross-architecture variability in projected excess deaths, underscoring the importance of model validation strategy and uncertainty quantification for public health decision-making. It also highlights practical deployment considerations, including sub-national extensions and dashboard tools for state health departments, aiming to improve resilience of public health planning against future shocks.

Abstract

Substance overdose mortality in the United States claimed over 80,000 lives in 2023, with the COVID-19 pandemic exacerbating existing trends through healthcare disruptions and behavioral changes. Estimating excess mortality, defined as deaths beyond expected levels based on pre-pandemic patterns, is essential for understanding pandemic impacts and informing intervention strategies. However, traditional statistical methods like SARIMA assume linearity, stationarity, and fixed seasonality, which may not hold under structural disruptions. We present a systematic comparison of SARIMA against three deep learning (DL) architectures (LSTM, Seq2Seq, and Transformer) for counterfactual mortality estimation using national CDC data (2015-2019 for training/validation, 2020-2023 for projection). We contribute empirical evidence that LSTM achieves superior point estimation (17.08% MAPE vs. 23.88% for SARIMA) and better-calibrated uncertainty (68.8% vs. 47.9% prediction interval coverage) when projecting under regime change. We also demonstrate that attention-based models (Seq2Seq, Transformer) underperform due to overfitting to historical means rather than capturing emergent trends. Ourreproducible pipeline incorporates conformal prediction intervals and convergence analysis across 60+ trials per configuration, and we provide an open-source framework deployable with 15 state health departments. Our findings establish that carefully validated DL models can provide more reliable counterfactual estimates than traditional methods for public health planning, while highlighting the need for calibration techniques when deploying neural forecasting in high-stakes domains.

Statistical vs. Deep Learning Models for Estimating Substance Overdose Excess Mortality in the US

TL;DR

This study tackles excess mortality estimation from US substance overdoses during the COVID-19 era by benchmarking traditional SARIMA against deep learning architectures (LSTM, Seq2Seq, Transformer) for counterfactual projection using CDC WONDER data. Through pre-pandemic validation and conformal prediction for uncertainty, it finds that LSTM provides the best balance of point accuracy and calibrated prediction intervals, while attention-based models overfit to historical means and SARIMA undercaptures nonlinear regime changes. The work delivers a reproducible open-source pipeline and demonstrates substantial cross-architecture variability in projected excess deaths, underscoring the importance of model validation strategy and uncertainty quantification for public health decision-making. It also highlights practical deployment considerations, including sub-national extensions and dashboard tools for state health departments, aiming to improve resilience of public health planning against future shocks.

Abstract

Substance overdose mortality in the United States claimed over 80,000 lives in 2023, with the COVID-19 pandemic exacerbating existing trends through healthcare disruptions and behavioral changes. Estimating excess mortality, defined as deaths beyond expected levels based on pre-pandemic patterns, is essential for understanding pandemic impacts and informing intervention strategies. However, traditional statistical methods like SARIMA assume linearity, stationarity, and fixed seasonality, which may not hold under structural disruptions. We present a systematic comparison of SARIMA against three deep learning (DL) architectures (LSTM, Seq2Seq, and Transformer) for counterfactual mortality estimation using national CDC data (2015-2019 for training/validation, 2020-2023 for projection). We contribute empirical evidence that LSTM achieves superior point estimation (17.08% MAPE vs. 23.88% for SARIMA) and better-calibrated uncertainty (68.8% vs. 47.9% prediction interval coverage) when projecting under regime change. We also demonstrate that attention-based models (Seq2Seq, Transformer) underperform due to overfitting to historical means rather than capturing emergent trends. Ourreproducible pipeline incorporates conformal prediction intervals and convergence analysis across 60+ trials per configuration, and we provide an open-source framework deployable with 15 state health departments. Our findings establish that carefully validated DL models can provide more reliable counterfactual estimates than traditional methods for public health planning, while highlighting the need for calibration techniques when deploying neural forecasting in high-stakes domains.
Paper Structure (39 sections, 8 figures, 7 tables)

This paper contains 39 sections, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation and Approach
  3. Contributions
  4. Related Work
  5. Methodology
  6. Data Sources and Extraction
  7. Dataset Split and Validation Strategy
  8. Models
  9. SARIMA (Baseline)
  10. LSTM
  11. Seq2Seq
  12. Transformer
  13. Hyperparameter Tuning
  14. Uncertainty Quantification
  15. Evaluation Metrics
  16. ...and 24 more sections

Figures (8)

  • Figure 1: Counterfactual mortality projections comparing SARIMA (baseline) against four DL architectures. Black line shows reported monthly substance overdose deaths; colored lines show model projections; shaded regions indicate 95% prediction/confidence intervals. Vertical dashed line (January 2020) marks projection start. LSTM (a) tracks the rising trend visible in late 2019, while attention-based models (b-d) project flat trajectories that underestimate observed mortality.
  • Figure 2: Prototype mortality projection dashboard interface. Users select models, projection horizons, and training windows through an interactive web interface. Planned extensions include state/county-level analysis, automatic CDC WONDER data ingestion, and scenario simulation for policy planning.
  • Figure 3: Convergence of LSTM validation metrics as a function of number of training trials. Point estimates (means) stabilize around 30-40 trials, but confidence interval widths (shaded regions) continue decreasing through 80-100 trials. Error bars show 95% CIs computed via bootstrap resampling.
  • Figure 4: 95% confidence interval width decay for each evaluation metric, plotted on log scale. Width decreases following expected $\mathcal{O}(1/\sqrt{N})$ rate, plateauing around 80-100 trials. RMSE and MAE show fastest convergence; PI coverage requires more trials to stabilize due to tail sensitivity.
  • Figure 5: LSTM counterfactual projections at increasing horizons. Uncertainty bounds (shaded regions) appropriately widen with projection length while maintaining coverage. Projection trajectories remain stable across horizons, indicating robust extrapolation behavior.
  • ...and 3 more figures