Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flusion: Integrating multiple data sources for accurate influenza predictions

Evan L. Ray, Yijin Wang, Russell D. Wolfinger, Nicholas G. Reich

TL;DR

Flusion tackles the challenge of forecasting influenza hospital admissions when the target NHSN history is limited by integrating signals with longer histories (FluSurv-NET and ILI+) through a joint, multi-location gradient boosting framework and a Bayesian autoregressive component. The model, combining GBQR, GBQR-no-level, and ARX with quantile averaging, achieves top performance in the 2023/24 FluSight season, driven mainly by cross-signal and cross-location learning. Post hoc analyses show that joint training and the GBQR component are key contributors, while certain preprocessing choices have nuanced effects. The work demonstrates the practical value of borrowing information across signals and locations for public health forecasting under data constraints, with implications for data modernization and transfer learning in surveillance systems.

Abstract

Over the last ten years, the US Centers for Disease Control and Prevention (CDC) has organized an annual influenza forecasting challenge with the motivation that accurate probabilistic forecasts could improve situational awareness and yield more effective public health actions. Starting with the 2021/22 influenza season, the forecasting targets for this challenge have been based on hospital admissions reported in the CDC's National Healthcare Safety Network (NHSN) surveillance system. Reporting of influenza hospital admissions through NHSN began within the last few years, and as such only a limited amount of historical data are available for this signal. To produce forecasts in the presence of limited data for the target surveillance system, we augmented these data with two signals that have a longer historical record: 1) ILI+, which estimates the proportion of outpatient doctor visits where the patient has influenza; and 2) rates of laboratory-confirmed influenza hospitalizations at a selected set of healthcare facilities. Our model, Flusion, is an ensemble that combines gradient boosting quantile regression models with a Bayesian autoregressive model. The gradient boosting models were trained on all three data signals, while the autoregressive model was trained on only the target signal; all models were trained jointly on data for multiple locations. Flusion was the top-performing model in the CDC's influenza prediction challenge for the 2023/24 season. In this article we investigate the factors contributing to Flusion's success, and we find that its strong performance was primarily driven by the use of a gradient boosting model that was trained jointly on data from multiple surveillance signals and locations. These results indicate the value of sharing information across locations and surveillance signals, especially when doing so adds to the pool of available training data.

Flusion: Integrating multiple data sources for accurate influenza predictions

TL;DR

Flusion tackles the challenge of forecasting influenza hospital admissions when the target NHSN history is limited by integrating signals with longer histories (FluSurv-NET and ILI+) through a joint, multi-location gradient boosting framework and a Bayesian autoregressive component. The model, combining GBQR, GBQR-no-level, and ARX with quantile averaging, achieves top performance in the 2023/24 FluSight season, driven mainly by cross-signal and cross-location learning. Post hoc analyses show that joint training and the GBQR component are key contributors, while certain preprocessing choices have nuanced effects. The work demonstrates the practical value of borrowing information across signals and locations for public health forecasting under data constraints, with implications for data modernization and transfer learning in surveillance systems.

Abstract

Over the last ten years, the US Centers for Disease Control and Prevention (CDC) has organized an annual influenza forecasting challenge with the motivation that accurate probabilistic forecasts could improve situational awareness and yield more effective public health actions. Starting with the 2021/22 influenza season, the forecasting targets for this challenge have been based on hospital admissions reported in the CDC's National Healthcare Safety Network (NHSN) surveillance system. Reporting of influenza hospital admissions through NHSN began within the last few years, and as such only a limited amount of historical data are available for this signal. To produce forecasts in the presence of limited data for the target surveillance system, we augmented these data with two signals that have a longer historical record: 1) ILI+, which estimates the proportion of outpatient doctor visits where the patient has influenza; and 2) rates of laboratory-confirmed influenza hospitalizations at a selected set of healthcare facilities. Our model, Flusion, is an ensemble that combines gradient boosting quantile regression models with a Bayesian autoregressive model. The gradient boosting models were trained on all three data signals, while the autoregressive model was trained on only the target signal; all models were trained jointly on data for multiple locations. Flusion was the top-performing model in the CDC's influenza prediction challenge for the 2023/24 season. In this article we investigate the factors contributing to Flusion's success, and we find that its strong performance was primarily driven by the use of a gradient boosting model that was trained jointly on data from multiple surveillance signals and locations. These results indicate the value of sharing information across locations and surveillance signals, especially when doing so adds to the pool of available training data.
Paper Structure (19 sections, 5 equations, 4 figures, 3 tables)

This paper contains 19 sections, 5 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Data Sources
  3. The FluSight collaborative forecasting exercise
  4. Notation and evaluation metrics
  5. Model
  6. Data standarization
  7. Component Models 1 and 2: GBQR and GBQR-no-level
  8. Component Model 3: ARX
  9. Ensemble Methods: Quantile averaging
  10. Model adjustments used for real time forecasts
  11. Software availability and reproducibility
  12. Real-time performance: the 2023/24 FluSight season
  13. Evaluation setup
  14. Evaluation results
  15. Post hoc model exploration
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Influenza data at the national level in the US. The top panel shows weekly hospital admissions from NHSN, including the 2023/24 season that was the target season for predictions described in this article. In the second panel, a dashed orange line shows raw data reported from FluSurv-NET; the modeled data, in blue, are obtained by scaling up the raw data using per-season inflation factors designed to account for varying testing rates and test sensitivity. In the third panel, a dashed orange line shows raw ILI data from ILINet; the modeled data in blue are ILI+ values obtained by combining ILI with test positivity rates. Dark grey shaded regions indicate pandemic seasons that were not used for model training; these include the 2008/09 and 2009/10 seasons which were impacted by pandemic H1N1 influenza, and the 2020/21 and 2021/22 seasons which were impacted by low influenza activity during the COVID pandemic. Light grey shaded regions indicate the off-season, which was not used for training the GBQR and GBQR-no-level models. Additionally, FluSurv-NET and ILI+ data for the 2023/24 season were not used for model training in this work.
  • Figure 2: Influenza data for all surveillance signals and all locations available for each data source after standardizing transformations have been applied. The top row shows weekly hospital admissions from NHSN, the second row shows data from FluSurv-NET, and the third row shows ILI+. The left column shows all state-level locations, while the right column shows national level data, as well as data at the level of HHS regions for the ILI+ signal. The horizontal axis is the season week. We define the season to begin on US Epidemic Week 31, which generally falls in early August; the range of season weeks shown corresponds approximately to the active flu season. Within each panel, there is one line for each combination of season and location for all seasons and locations that are available for the given surveillance system at the state, regional, and national levels. Line color corresponds to the population size of the location; the darkest lines are for the national level while the lightest lines are for states with small populations.
  • Figure 3: Influenza data and forecasts for the six states with the largest cumulative hospital admissions during the 2023/24 season. To avoid overplotting, in this figure forecasts from every fourth reference date are shown; evaluations include all reference dates. Forecasts are represented by the predictive median (black lines) and 50% and 95% prediction intervals (blue shaded regions). Solid orange lines show the finalized admission counts reported as of May 1, 2024, while dotted orange lines show the initial reported values that were available on the date predictions were generated.
  • Figure 4: Influenza data and evaluation results. Panel (a): Weekly influenza hospital admissions reported in NHSN for the 2023/24 season, aggregated across all forecasted state-level locations. Panel (b): rMWIS for models contributing to FluSight, by forecast horizon (panels) and target date (horizontal axis). Lower rMWIS indicates better forecast performance. rMWIS values greater than 2.5 are not displayed. Panel (c): One-sided quantile coverage differential, computed as empirical coverage rate minus nominal coverage rate. A well-calibrated model has a differential of 0, while a conservative method (with wide prediction intervals) has a negative differential at nominal coverage rates less than 0.5 and a positive differential at nominal coverage rates greater than 0.5, indicated with blue shading.