MATA-Former & SIICU: Semantic Aware Temporal Alignment for High-Fidelity ICU Risk Prediction

Abstract

Forecasting evolving clinical risks relies on intrinsic pathological dependencies rather than mere chronological proximity, yet current methods struggle with coarse binary supervision and physical timestamps. To align predictive modeling with clinical logic, we propose the Medical-semantics Aware Time-ALiBi Transformer (MATA-Former), utilizing event semantics to dynamically parameterize attention weights to prioritize causal validity over time lags. Furthermore, we introduce Plateau-Gaussian Soft Labeling (PSL), reformulating binary classification into continuous multi-horizon regression for full-trajectory risk modeling. Evaluated on SIICU -- a newly constructed dataset featuring over 506k events with rigorous expert-verified, fine-grained annotations -- and the MIMIC-IV dataset, our framework demonstrates superior efficacy and robust generalization in capturing risks from text-intensive, irregular clinical time series.

Figures (9)

• Figure 1: Statistics of SIICU. (a) Heavy-tailed distribution of sequence lengths. (b) Irregular time intervals. (c) Density of positive sample ratios. (d) Long-tail distribution of risks. These distributions highlight the dataset's high heterogeneity and sparsity.
• Figure 2: Overview of the SIICU & MATA-Former framework. (A) The SIICU pipeline transforms heterogeneous clinical records into chronological timelines, utilizing expert-verified collaborative annotation for ground truth generation. (B) The MATA-Former Architecture integrates semantic embeddings with log-transformed temporal features. The core MATA mechanism modulates self-attention via dynamic Laplacian biases, optimized against PSLs for multi-horizon risk prediction.
• Figure 3: Ablation and mechanism analysis. (a) Additive bias superiorly preserves semantic integrity compared to temporal encoding. (b)-(c) MSE maximizes AUPRC by maintaining a distinct signal-noise separation valley. (d) Decoupling $\mu$ and $\alpha$ achieves optimality in capturing heterogeneous clinical dynamics.
• Figure 4: Visualization of learnable temporal dynamics. (a) Dynamic Parameter Evolution: The distributional shift of Laplacian parameters ($\mu, \alpha$) from Layer 0 to 5 reveals a transition from discrete temporal filtering to high-level semantic abstraction. (b) Aggregated Attention Alignment: To accommodate extended clinical trajectories, the heatmap utilizes $30 \times 30$ block-wise average pooling. The resulting vertical strata demonstrate consistent anchoring on pivotal prognostic determinants—specifically abnormal labs and interventional history—validating the structural alignment between the Laplacian prior and clinical causality.
• Figure 4: Representative sampling points of $\mu_{i}^{(h)}$ within the search space and their corresponding clinical interpretations.