Integration of Large Language Models and Traditional Deep Learning for Social Determinants of Health Prediction

TL;DR

This work tackles automatic extraction of social determinants of health from clinical text by systematically comparing traditional deep learning and large language model approaches on public datasets, and by introducing a hybrid two-step classifier that preserves high accuracy while dramatically reducing latency. By leveraging a binary detector from traditional DL and a precision-focused LLM for multilabel labeling, the approach achieves substantial speedups (up to ~73× faster than the largest LLM in some setups) with competitive macro F1 performance, and benefits further from synthetic data augmentation (Amended dataset). The results show that LLMs deliver stronger performance on mimic data, while traditional DL with engineered features can be highly stable with synthetic augmentation, suggesting a practical path for real-world SDoH prediction that balances accuracy, latency, and data efficiency. The study provides detailed methodology, prompts, and data splits to enable reproducibility and future improvements in SDoH extraction from clinical narratives.

Abstract

Social Determinants of Health (SDoH) are economic, social and personal circumstances that affect or influence an individual's health status. SDoHs have shown to be correlated to wellness outcomes, and therefore, are useful to physicians in diagnosing diseases and in decision-making. In this work, we automatically extract SDoHs from clinical text using traditional deep learning and Large Language Models (LLMs) to find the advantages and disadvantages of each on an existing publicly available dataset. Our models outperform a previous reference point on a multilabel SDoH classification by 10 points, and we present a method and model to drastically speed up classification (12X execution time) by eliminating expensive LLM processing. The method we present combines a more nimble and efficient solution that leverages the power of the LLM for precision and traditional deep learning methods for efficiency. We also show highly performant results on a dataset supplemented with synthetic data and several traditional deep learning models that outperform LLMs. Our models and methods offer the next iteration of automatic prediction of SDoHs that impact at-risk patients.