Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models

TL;DR

Masked autoencoders for EHRs typically use uniform masking, which underutilizes the learning signal from volatile laboratory biomarkers. The authors introduce Coefficient of Variation Masking (CV-Masking) implemented in a Value-Only Masked Autoencoder (VO-MAE) that biases masking toward volatile biomarkers using the coefficient of variation and a 75th percentile threshold, with a small unmasked regularization term. Across 100 high-frequency labs in MIMIC-IV, CV-Masking improves reconstruction, enhances downstream predictions (mortality and readmission), and accelerates convergence by up to 50%. Perturbation analyses show CV-based representations rely more on patient-specific temporal context, supporting deeper, more clinically meaningful learning. This work demonstrates a principled, domain-informed pretraining strategy that improves robustness and efficiency of EHR foundation representations.

Abstract

Masked autoencoders (MAEs) are increasingly applied to electronic health records (EHR) for learning general-purpose representations that support diverse clinical tasks. However, existing approaches typically rely on uniform random masking, implicitly assuming all features are equally predictable. In reality, laboratory tests exhibit substantial heterogeneity in volatility: some biomarkers (e.g., sodium) remain stable, while others (e.g., lactate) fluctuate considerably and are more difficult to model. Clinically, volatile biomarkers often signal acute pathophysiology and require more sophisticated modeling to capture their complex temporal patterns. We propose a volatility-aware pretraining strategy, Coefficient of Variation Masking (CV-Masking), that adaptively adjusts masking probabilities according to the intrinsic variability of each feature. Combined with a value-only masking objective aligned with clinical workflows, CV-Masking yields systematic improvements over random and variance-based strategies. Experiments on a large panel of laboratory tests show that CV-Masking enhances reconstruction, improves downstream predictive performance, and accelerates convergence, producing more robust and clinically meaningful EHR representations.

Figures (9)

• Figure 1: Top-20 Laboratory Improvements from CV-Based Masking. CV-masking yields largest gains for immune markers (Lymphocytes +0.32, Basophils +0.20), liver function tests (ALT +0.19), and metabolic indicators (CO2 +0.16). Green labels show absolute R² improvement values.
• Figure 2: Value-only Masked Autoencoder for MEDS triplets. Only the value is masked while time and code remain visible. The encoder consumes full-context triplets; the decoder reconstructs masked values via cross-attention.
• Figure 3: Distribution of Performance Improvements. CV-based masking achieves systematic wins across 71% of laboratory tests (green), with particularly strong gains (R² > 0.1) in 15% of cases.
• Figure 4: CV-based masking shows 2.1× greater sensitivity to corrupted historical context, indicating deeper temporal learning.
• Figure 5: Training Convergence Analysis. CV-based masking achieves faster and more stable convergence than random and variance-based masking. These curves demonstrate that performance gains stem from the CV-masking strategy itself under controlled conditions, not from differential training durations or hyperparameter configurations.