Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structured Insight from Unstructured Data: Large Language Models for SDOH-Driven Diabetes Risk Prediction

Sasha Ronaghi, Prerit Choudhary, David H Rehkopf, Bryant Lin

TL;DR

This paper investigates whether large language models (LLMs) can derive structured social determinants of health (SDOH) from unstructured patient life stories and whether these narratives and derived features improve diabetes risk prediction. Using 65 older adults, the authors extract 16 SDOH risk factors across 5 topics via retrieval-augmented generation and convert them into quantitative ratings, then integrate these with standard laboratory biomarkers in Ridge, Lasso, Random Forest, and XGBoost models. They also test LLMs’ ability to predict A1C control directly from interview transcripts with the A1C values redacted, achieving up to 60% accuracy with GPT-4o. The results show modest predictive value for SDOH features in small datasets and highlight the potential and challenges of incorporating unstructured narratives into clinical risk workflows, emphasizing the need for larger, longitudinal studies and physician validation. This approach points to a scalable path for enriching diabetes risk models with contextual patient data and could extend to other chronic diseases with broader implementation considerations.

Abstract

Social determinants of health (SDOH) play a critical role in Type 2 Diabetes (T2D) management but are often absent from electronic health records and risk prediction models. Most individual-level SDOH data is collected through structured screening tools, which lack the flexibility to capture the complexity of patient experiences and unique needs of a clinic's population. This study explores the use of large language models (LLMs) to extract structured SDOH information from unstructured patient life stories and evaluate the predictive value of both the extracted features and the narratives themselves for assessing diabetes control. We collected unstructured interviews from 65 T2D patients aged 65 and older, focused on their lived experiences, social context, and diabetes management. These narratives were analyzed using LLMs with retrieval-augmented generation to produce concise, actionable qualitative summaries for clinical interpretation and structured quantitative SDOH ratings for risk prediction modeling. The structured SDOH ratings were used independently and in combination with traditional laboratory biomarkers as inputs to linear and tree-based machine learning models (Ridge, Lasso, Random Forest, and XGBoost) to demonstrate how unstructured narrative data can be applied in conventional risk prediction workflows. Finally, we evaluated several LLMs on their ability to predict a patient's level of diabetes control (low, medium, high) directly from interview text with A1C values redacted. LLMs achieved 60% accuracy in predicting diabetes control levels from interview text. This work demonstrates how LLMs can translate unstructured SDOH-related data into structured insights, offering a scalable approach to augment clinical risk models and decision-making.

Structured Insight from Unstructured Data: Large Language Models for SDOH-Driven Diabetes Risk Prediction

TL;DR

This paper investigates whether large language models (LLMs) can derive structured social determinants of health (SDOH) from unstructured patient life stories and whether these narratives and derived features improve diabetes risk prediction. Using 65 older adults, the authors extract 16 SDOH risk factors across 5 topics via retrieval-augmented generation and convert them into quantitative ratings, then integrate these with standard laboratory biomarkers in Ridge, Lasso, Random Forest, and XGBoost models. They also test LLMs’ ability to predict A1C control directly from interview transcripts with the A1C values redacted, achieving up to 60% accuracy with GPT-4o. The results show modest predictive value for SDOH features in small datasets and highlight the potential and challenges of incorporating unstructured narratives into clinical risk workflows, emphasizing the need for larger, longitudinal studies and physician validation. This approach points to a scalable path for enriching diabetes risk models with contextual patient data and could extend to other chronic diseases with broader implementation considerations.

Abstract

Social determinants of health (SDOH) play a critical role in Type 2 Diabetes (T2D) management but are often absent from electronic health records and risk prediction models. Most individual-level SDOH data is collected through structured screening tools, which lack the flexibility to capture the complexity of patient experiences and unique needs of a clinic's population. This study explores the use of large language models (LLMs) to extract structured SDOH information from unstructured patient life stories and evaluate the predictive value of both the extracted features and the narratives themselves for assessing diabetes control. We collected unstructured interviews from 65 T2D patients aged 65 and older, focused on their lived experiences, social context, and diabetes management. These narratives were analyzed using LLMs with retrieval-augmented generation to produce concise, actionable qualitative summaries for clinical interpretation and structured quantitative SDOH ratings for risk prediction modeling. The structured SDOH ratings were used independently and in combination with traditional laboratory biomarkers as inputs to linear and tree-based machine learning models (Ridge, Lasso, Random Forest, and XGBoost) to demonstrate how unstructured narrative data can be applied in conventional risk prediction workflows. Finally, we evaluated several LLMs on their ability to predict a patient's level of diabetes control (low, medium, high) directly from interview text with A1C values redacted. LLMs achieved 60% accuracy in predicting diabetes control levels from interview text. This work demonstrates how LLMs can translate unstructured SDOH-related data into structured insights, offering a scalable approach to augment clinical risk models and decision-making.
Paper Structure (12 sections, 5 figures)

This paper contains 12 sections, 5 figures.

Table of Contents

  1. INTRODUCTION
  2. METHODS
  3. Extracting structured, free-form SDOH data
  4. Using extracted SDOH data for risk prediction
  5. Predicting level of diabetes control
  6. RESULTS AND DISCUSSION
  7. Extracting structured, free-form SDOH data
  8. Using extracted SDOH data for risk prediction
  9. Predicting level of diabetes control
  10. CONCLUSIONS
  11. LLM prompts
  12. Scale for Rating Patients on SDOH factors

Figures (5)

  • Figure 1: Percentage distribution of how frequently a specific topic was discussed across patient interviews.
  • Figure 2: LLM generated summary for "Social Support - Social Networks" analysis for one patient
  • Figure 3: Training and cross‐validated $R^{2}$ by model and feature set.
  • Figure 4: Feature Importance for Combined Feature Set, Random-Forest
  • Figure 5: Categorical and overall accuracy for each model across 65 patients. o1-mini did not provide a response for 4 patients.