Chronic Kidney Disease Prognosis Prediction Using Transformer

TL;DR

CKD prognosis prediction benefits from modeling long-range temporal patterns in multi-domain EHR data. The authors introduce ProQ-BERT, a transformer-based framework that tokenizes demographic, clinical, and laboratory information, including quantized lab values, and is pretrained with masked language modeling before fine-tuning for prognosis binaries. Evaluated on 91,816 patients from the SNUH OMOP CDM, ProQ-BERT consistently outperforms a CEHR-BERT baseline, achieving ROC-AUC up to 0.995 and PR-AUC up to 0.989 in short-term predictions and maintaining strong performance over longer horizons. The work demonstrates the value of temporal design and lab quantization for CKD care, offering a scalable approach to personalized prognosis with interpretable attention mechanisms.

Abstract

Chronic Kidney Disease (CKD) affects nearly 10\% of the global population and often progresses to end-stage renal failure. Accurate prognosis prediction is vital for timely interventions and resource optimization. We present a transformer-based framework for predicting CKD progression using multi-modal electronic health records (EHR) from the Seoul National University Hospital OMOP Common Data Model. Our approach (\textbf{ProQ-BERT}) integrates demographic, clinical, and laboratory data, employing quantization-based tokenization for continuous lab values and attention mechanisms for interpretability. The model was pretrained with masked language modeling and fine-tuned for binary classification tasks predicting progression from stage 3a to stage 5 across varying follow-up and assessment periods. Evaluated on a cohort of 91,816 patients, our model consistently outperformed CEHR-BERT, achieving ROC-AUC up to 0.995 and PR-AUC up to 0.989 for short-term prediction. These results highlight the effectiveness of transformer architectures and temporal design choices in clinical prognosis modeling, offering a promising direction for personalized CKD care.

Figures (2)

• Figure 1: Cohort Definition Flowchart
• Figure 2: Patient sequence tokenization process showing demographic tokens, visit segments, medical events with temporal relationships, and quantized laboratory measurements.