A Comparative Study of Transformer-Based Models for Multi-Horizon Blood Glucose Prediction

TL;DR

This study addresses blood glucose forecasting for type 1 diabetes using transformer-based multivariate time-series across multi-horizon forecasts up to 4 hours and history up to 1 week. It systematically compares embedding schemes—point-wise, patch-wise, series-wise, and hybrid—across architectures including Vanilla Transformer, Crossformer, PatchTST, iTransformer, and TimeXer, against baselines ZOH and DLinear, on the DCLP-3 and OhioT1DM datasets. Key findings show Crossformer excels at 30-minute predictions with a one-week history, while PatchTST dominates longer horizons (120–240 minutes), with DLinear generally underperforming and the plain Transformer lacking generalization on external data. The results illustrate transformers’ ability to capture seasonal and long-range dependencies in multivariate BG data, informing model design choices for accurate, clinically useful glucose forecasting. The work also highlights avenues for improving efficiency and interpretability in transformer-based BG prediction systems.

Abstract

Accurate blood glucose prediction can enable novel interventions for type 1 diabetes treatment, including personalized insulin and dietary adjustments. Although recent advances in transformer-based architectures have demonstrated the power of attention mechanisms in complex multivariate time series prediction, their potential for blood glucose (BG) prediction remains underexplored. We present a comparative analysis of transformer models for multi-horizon BG prediction, examining forecasts up to 4 hours and input history up to 1 week. The publicly available DCLP3 dataset (n=112) was split (80%-10%-10%) for training, validation, and testing, and the OhioT1DM dataset (n=12) served as an external test set. We trained networks with point-wise, patch-wise, series-wise, and hybrid embeddings, using CGM, insulin, and meal data. For short-term blood glucose prediction, Crossformer, a patch-wise transformer architecture, achieved a superior 30-minute prediction of RMSE (15.6 mg / dL on OhioT1DM). For longer-term predictions (1h, 2h, and 4h), PatchTST, another path-wise transformer, prevailed with the lowest RMSE (24.6 mg/dL, 36.1 mg/dL, and 46.5 mg/dL on OhioT1DM). In general, models that used tokenization through patches demonstrated improved accuracy with larger input sizes, with the best results obtained with a one-week history. These findings highlight the promise of transformer-based architectures for BG prediction by capturing and leveraging seasonal patterns in multivariate time-series data to improve accuracy.

Figures (2)

• Figure 1: Illustration of the transformer models architecture for blood glucose prediction.
• Figure 2: Comparison of RMSE across models and prediction horizons for (a) OhioT1DM and (b) DCLP3 datasets. Each subfigure highlights the impact of history length and prediction horizon on model performance.