Early Prediction of Type 2 Diabetes Using Multimodal data and Tabular Transformers

TL;DR

This work tackles early Type 2 Diabetes risk prediction by leveraging a multimodal dataset that combines DXA bone measurements with electronic health records from the Qatar Biobank. It introduces TabTrans, a tabular transformer designed to model longitudinal, heterogeneous tabular data and captures long-range interactions that traditional methods may miss. The study performs extensive feature engineering on DXA-derived metrics, compares TabTrans against conventional ML and large language models, and provides probabilistic interpretations highlighting visceral adiposity as a key predictor. While TabTrans demonstrates strong performance against generative AI models, some conventional ML approaches show competitive or superior AUC in this dataset, underscoring the value of traditional methods on smaller tabular datasets. The findings support the potential of transformer-based tabular models for proactive, personalized diabetes management in specific populations, with clear DXA-derived indicators guiding risk stratification and intervention.

Abstract

This study introduces a novel approach for early Type 2 Diabetes Mellitus (T2DM) risk prediction using a tabular transformer (TabTrans) architecture to analyze longitudinal patient data. By processing patients` longitudinal health records and bone-related tabular data, our model captures complex, long-range dependencies in disease progression that conventional methods often overlook. We validated our TabTrans model on a retrospective Qatar BioBank (QBB) cohort of 1,382 subjects, comprising 725 men (146 diabetic, 579 healthy) and 657 women (133 diabetic, 524 healthy). The study integrated electronic health records (EHR) with dual-energy X-ray absorptiometry (DXA) data. To address class imbalance, we employed SMOTE and SMOTE-ENN resampling techniques. The proposed model`s performance is evaluated against conventional machine learning (ML) and generative AI models, including Claude 3.5 Sonnet (Anthropic`s constitutional AI), GPT-4 (OpenAI`s generative pre-trained transformer), and Gemini Pro (Google`s multimodal language model). Our TabTrans model demonstrated superior predictive performance, achieving ROC AUC $\geq$ 79.7 % for T2DM prediction compared to both generative AI models and conventional ML approaches. Feature interpretation analysis identified key risk indicators, with visceral adipose tissue (VAT) mass and volume, ward bone mineral density (BMD) and bone mineral content (BMC), T and Z-scores, and L1-L4 scores emerging as the most important predictors associated with diabetes development in Qatari adults. These findings demonstrate the significant potential of TabTrans for analyzing complex tabular healthcare data, providing a powerful tool for proactive T2DM management and personalized clinical interventions in the Qatari population. Index Terms: tabular transformers, multimodal data, DXA data, diabetes, T2DM, feature interpretation, tabular data

Figures (6)

• Figure 1: Systematic methodology followed to perform this research work.
• Figure 2: Feature reduction using PCA.
• Figure 3: Cluster-based analysis of important features
• Figure 4: TabTrans performance results for diabetes prediction using multimodal QBB data.
• Figure 5: Typical ML performance results for diabetes prediction using multimodal QBB data.