MedFuse: Multiplicative Embedding Fusion For Irregular Clinical Time Series

TL;DR

MedFuse tackles the challenge of irregular clinical time series by introducing MuFuse, a value-conditioned multiplicative fusion module that modulates feature embeddings with observed values. This imputation-free approach preserves feature identity while enabling nonlinear, feature-specific interactions, improving predictive performance across ICU mortality and chronic disease onset tasks. Empirical results on three real-world datasets show consistent gains over strong baselines, with ablations confirming the advantage of multiplicative fusion, and cross-dataset transfer experiments demonstrating the potential for pretraining and adaptation. The work advances practical modeling of EHR data by providing a general, scalable framework that captures clinically meaningful value–feature interactions and supports transfer learning across heterogeneous healthcare cohorts.

Abstract

Clinical time series derived from electronic health records (EHRs) are inherently irregular, with asynchronous sampling, missing values, and heterogeneous feature dynamics. While numerical laboratory measurements are highly informative, existing embedding strategies usually combine feature identity and value embeddings through additive operations, which constrains their ability to capture value-dependent feature interactions. We propose MedFuse, a framework for irregular clinical time series centered on the MuFuse (Multiplicative Embedding Fusion) module. MuFuse fuses value and feature embeddings through multiplicative modulation, preserving feature-specific information while modeling higher-order dependencies across features. Experiments on three real-world datasets covering both intensive and chronic care show that MedFuse consistently outperforms state-of-the-art baselines on key predictive tasks. Analysis of the learned representations further demonstrates that multiplicative fusion enhances expressiveness and supports cross-dataset pretraining. These results establish MedFuse as a generalizable approach for modeling irregular clinical time series.

Figures (7)

• Figure 1: Illustration of EVAT.
• Figure 2: MedFuse architecture with MuFuse value–feature fusion. Numerical observations $(f,v,t)$ are embedded by MuFuse: a feature embedder maps the identity $f\!\to\!\mathbf{e}_f\!\in\!\mathbb{R}^d$, a non-linear projector maps the measured value $v\!\to\!\mathbf{e}_v\!\in\!\mathbb{R}^d$, and the token embedding is formed by element-wise (broadcasted) Hadamard product $\mathbf{e}_{f,v}=\mathbf{e}_f\odot\mathbf{e}_v$, followed by adding a time/positional encoding to yield $\mathbf{e}_{f,v,t}$. Categorical events use a categorical embedder to produce $\mathbf{e}_{f,c,t}$. All tokens are flattened into a sequence with a missing-value mask and processed by an $N$-layer Transformer encoder; a linear softmax head outputs the target distribution. By modulating $\mathbf{e}_f$ multiplicatively with the observed value, MuFuse preserves feature identity while enabling rich, value-dependent interactions.
• Figure 3: Comparison of different partitioning factors $k$ on P12.
• Figure 4: TSNE maaten2008visualizing visualization of MedFuse's $\mathbf{e}_{f,v}$ on the HCC dataset before (Left) and after (Right) passing through the first layer of the Transformer encoder. Each point represents a token's embedding, colored by its feature type. We can clearly see that MedFuse successfully embeds tokens with the same feature type into a cluster. This characteristic is preserved after passing the embedding through the first layer of the Transformer encoder, demonstrating MedFuse's robustness.
• Figure 5: Comparison of the time embedding fusion effect. (Left): Fused to the feature identity embedding through MuFuse (broadcasted Hadamard product). (Right): Fused to the feature identity embedding through addition.