Synthetic Clinical Notes for Rare ICD Codes: A Data-Centric Framework for Long-Tail Medical Coding
Truong Vo, Weiyi Wu, Kaize Ding
TL;DR
The paper tackles the challenge of automatic ICD coding under extreme long tail distributions by proposing a data centric framework that generates high quality synthetic discharge summaries for rare and zero shot codes. It constructs plausible multi label code sets anchored on rare codes through code frequency stratification, log inverse synthetic allocation, and anchor based code set construction, and guides generation with knowledge injection prompts using ICD descriptions, synonyms, and hierarchy. The authors create approximately 90,000 synthetic notes covering around 7,905 codes and fine tune two state of the art transformers, PLM-ICD and GKI-ICD, on the original and extended data. Results show modest but consistent improvements in macro metrics and topK precision, demonstrating that carefully crafted synthetic data can improve equity in long tail ICD code prediction, albeit with significant computational cost and limitations in fully resolving long tail challenges.
Abstract
Automatic ICD coding from clinical text is a critical task in medical NLP but remains hindered by the extreme long-tail distribution of diagnostic codes. Thousands of rare and zero-shot ICD codes are severely underrepresented in datasets like MIMIC-III, leading to low macro-F1 scores. In this work, we propose a data-centric framework that generates high-quality synthetic discharge summaries to mitigate this imbalance. Our method constructs realistic multi-label code sets anchored on rare codes by leveraging real-world co-occurrence patterns, ICD descriptions, synonyms, taxonomy, and similar clinical notes. Using these structured prompts, we generate 90,000 synthetic notes covering 7,902 ICD codes, significantly expanding the training distribution. We fine-tune two state-of-the-art transformer-based models, PLM-ICD and GKI-ICD, on both the original and extended datasets. Experiments show that our approach modestly improves macro-F1 while maintaining strong micro-F1, outperforming prior SOTA. While the gain may seem marginal relative to the computational cost, our results demonstrate that carefully crafted synthetic data can enhance equity in long-tail ICD code prediction.