Contrastive Learning on Medical Intents for Sequential Prescription Recommendation

TL;DR

This work addresses the challenge of modeling multiple coexisting temporal patterns in EHR data to improve next-visit prescription recommendations. It introduces ARCI, a multi-level transformer framework that learns intra-visit and inter-visit dependencies to form temporal paths, and uses a contrastive loss to align each transformer head with a distinct medical intent, enabling diverse and interpretable patient representations. Evaluations on MIMIC-III and AKI show ARCI outperforms state-of-the-art baselines on ranking and classification tasks, while providing clinically meaningful interpretability through attention-based temporal paths. The approach advances personalized, safer prescription recommendations by capturing multiple medical intents and temporal dependencies across visits.

Abstract

Recent advancements in sequential modeling applied to Electronic Health Records (EHR) have greatly influenced prescription recommender systems. While the recent literature on drug recommendation has shown promising performance, the study of discovering a diversity of coexisting temporal relationships at the level of medical codes over consecutive visits remains less explored. The goal of this study can be motivated from two perspectives. First, there is a need to develop a sophisticated sequential model capable of disentangling the complex relationships across sequential visits. Second, it is crucial to establish multiple and diverse health profiles for the same patient to ensure a comprehensive consideration of different medical intents in drug recommendation. To achieve this goal, we introduce Attentive Recommendation with Contrasted Intents (ARCI), a multi-level transformer-based method designed to capture the different but coexisting temporal paths across a shared sequence of visits. Specifically, we propose a novel intent-aware method with contrastive learning, that links specialized medical intents of the patients to the transformer heads for extracting distinct temporal paths associated with different health profiles. We conducted experiments on two real-world datasets for the prescription recommendation task using both ranking and classification metrics. Our results demonstrate that ARCI has outperformed the state-of-the-art prescription recommendation methods and is capable of providing interpretable insights for healthcare practitioners.

Figures (7)

• Figure 1: In EHR, a patient's sequence of visits often involves more than one thread of prescribing drugs driven by different medical intents (e.g., influenza and ADHD). We use temporal paths to distinguish the distinct dependencies among medical codes associated with different intents.
• Figure 2: In the ARCI framework, the sequence of visits is firstly processed within the Temporal Paths Representation module including the Visit-Level Transformer to extract intra-visit dependencies, and the Cross-Visit Transformer to generate temporal paths based on inter-visit codes dependency. Meanwhile, the input embeddings of each visit are passed through multiple linear layers to generate multiple medical intents, and a contrastive learning objective is further utilized to refine the attention heads in the Contrasted Intents module. Lastly, the output of the Temporal Paths Representation module and the intent representations are fed to a visit-instance attention mechanism inside the Aggregation and Prediction module.
• Figure 3: The Cross-Visit Transformer generates a temporal attention matrix, where prescription embeddings at time step $t$ are multiplied by the attention matrix and shared with prescriptions at time step $t+1$. The figure illustrates the temporal attention generation for a single prescription.
• Figure 4: t-SNE of the heads and intents. For better illustration, only the means of the intents' representations are shown as crosses. The figure depicts how different heads are successfully clustered into different groups that are aligned with their associated intents.
• Figure 5: The effect of varying numbers of heads and intents on the PRAUC Metric with and without $\mathcal{L}_{I}$ on the MIMIC-III and AKI datasets.