ChronoFormer: Time-Aware Transformer Architectures for Structured Clinical Event Modeling

TL;DR

ChronoFormer introduces a time-aware transformer for structured clinical data by embedding continuous-time information into the model and employing a hierarchical attention mechanism to capture both local and global temporal dependencies. The architecture combines temporally aware embeddings with a two-level attention scheme and a domain-specific pretraining objective, Masked Event Modeling (MEM), to learn clinically meaningful representations from MEDS-formatted EHR sequences. Empirically, ChronoFormer achieves state-of-the-art performance on mortality, readmission, and multi-label comorbidity tasks in MIMIC-IV and demonstrates strong cross-system generalization to eICU, highlighting its robustness and clinical relevance. The work suggests that time-aware, structured temporal models can offer substantial benefits over time-agnostic transformers and time-discretized approaches for real-world healthcare analytics.

Abstract

The temporal complexity of electronic health record (EHR) data presents significant challenges for predicting clinical outcomes using machine learning. This paper proposes ChronoFormer, an innovative transformer based architecture specifically designed to encode and leverage temporal dependencies in longitudinal patient data. ChronoFormer integrates temporal embeddings, hierarchical attention mechanisms, and domain specific masking techniques. Extensive experiments conducted on three benchmark tasks mortality prediction, readmission prediction, and long term comorbidity onset demonstrate substantial improvements over current state of the art methods. Furthermore, detailed analyses of attention patterns underscore ChronoFormer's capability to capture clinically meaningful long range temporal relationships.