Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Glucose Forecasting

TL;DR

The paper tackles BG forecasting from multimodal, irregularly sampled data with a focus on real-time edge deployment. It introduces GlucoNet, a hybrid architecture that a) decomposes BG signals with Variational Mode Decomposition, b) converts sparse event data (carbs, insulin) into continuous features via Sparse Signal Reconstruction, and c) uses a dual-branch forecasting strategy where low-frequency components are predicted by an LSTM and high-frequency components by a Transformer enhanced with Knowledge Distillation. Empirical results on the OhioT1DM dataset show substantial gains, including roughly 60% RMSE improvement and a 21% reduction in parameters, with MAE improvements around 51–57% across 12 participants. These findings indicate GlucoNet's potential as an accurate, compact tool for real-world glucose forecasting and proactive diabetes management on edge devices.

Abstract

The availability of continuous glucose monitors as over-the-counter commodities have created a unique opportunity to monitor a person's blood glucose levels, forecast blood glucose trajectories and provide automated interventions to prevent devastating chronic complications that arise from poor glucose control. However, forecasting blood glucose levels is challenging because blood glucose changes consistently in response to food intake, medication intake, physical activity, sleep, and stress. It is particularly difficult to accurately predict BGL from multimodal and irregularly sampled data and over long prediction horizons. Furthermore, these forecasting models must operate in real-time on edge devices to provide in-the-moment interventions. To address these challenges, we propose GlucoNet, an AI-powered sensor system for continuously monitoring behavioral and physiological health and robust forecasting of blood glucose patterns. GlucoNet devises a feature decomposition-based transformer model that incorporates patients' behavioral and physiological data and transforms sparse and irregular patient data (e.g., diet and medication intake data) into continuous features using a mathematical model, facilitating better integration with the BGL data. Given the non-linear and non-stationary nature of BG signals, we propose a decomposition method to extract both low and high-frequency components from the BGL signals, thus providing accurate forecasting. To reduce the computational complexity, we also propose to employ knowledge distillation to compress the transformer model. GlucoNet achieves a 60% improvement in RMSE and a 21% reduction in the number of parameters, improving RMSE and MAE by 51% and 57%, using data obtained involving 12 participants with T1-Diabetes. These results underscore GlucoNet's potential as a compact and reliable tool for real-world diabetes prevention and management.

Figures (7)

• Figure 1: Accuracy-efficiency trade-offs Martinsson et al.martinsson2020blood, CNN-RNN freiburghaus2020deep, GlySim arefeen2023glysim, CNN-RNN daniels2021multitask, and MTL-LSTM shuvo2023deep. RMSE is used to quantify the accuracy (quality) of the models, and the total parameters of the model are used to quantify the efficiency (RMSE vs. Total parameters).
• Figure 2: An Overview of GlucoNet includes sensing to measure variables, Sparse signal construction to extract the effective variables, Feature decomposition using the VMD method, Data stratification to combine various variables, and a forecasting module using LSTM and Knowledge Distillation Framework.
• Figure 3: An example of transforming sparse events (carb intake and insulin dosage) using Sparse Signal Reconstruction (SSR) module.
• Figure 4: Overall Transformer model architecture for the teacher model {$64$ input dimensions, $4$ attention heads, $128$ feed-forward units} and student model {$32$ input dimensions, $2$ attention heads, $64$ feed-forward units}.
• Figure 5: An example of forecasting the blood glucose model with $PH=60$ minutes for (a) low-frequency features, (b) high-frequency features, and (c) blood glucose levels for a participant of the Ohio dataset