A large language model-based approach to quantifying the effects of social determinants in liver transplant decisions

TL;DR

This work tackles how social determinants of health shape liver transplant decisions and contribute to systematic disparities. It introduces an LLM-based pipeline that extracts 23 SDOH factors from unstructured psychosocial notes to create standardized patient snapshots, enabling population-level and subpopulation analyses. The study demonstrates that SDOH snapshots meaningfully improve predictions of psychosocial recommendations and listing decisions, reveal temporal and demographic patterns, and quantify the explained portion of disparities via Blinder-Oaxaca decomposition. The approach offers actionable insights for targeted interventions and provides a generalizable framework for analyzing unstructured life-context data across other medical domains.

Abstract

Patient life circumstances, including social determinants of health (SDOH), shape both health outcomes and care access, contributing to persistent disparities across gender, race, and socioeconomic status. Liver transplantation exemplifies these challenges, requiring complex eligibility and allocation decisions where SDOH directly influence patient evaluation. We developed an artificial intelligence (AI)-driven framework to analyze how broadly defined SDOH -- encompassing both traditional social determinants and transplantation-related psychosocial factors -- influence patient care trajectories. Using large language models, we extracted 23 SDOH factors related to patient eligibility for liver transplantation from psychosocial evaluation notes. These SDOH ``snapshots'' significantly improve prediction of patient progression through transplantation evaluation stages and help explain liver transplantation decisions including the recommendation based on psychosocial evaluation and the listing of a patient for a liver transplantation. Our analysis helps identify patterns of SDOH prevalence across demographics that help explain racial disparities in liver transplantation decisions. We highlight specific unmet patient needs, which, if addressed, could improve the equity and efficacy of transplant care. While developed for liver transplantation, this systematic approach to analyzing previously unstructured information about patient circumstances and clinical decision-making could inform understanding of care decisions and disparities across various medical domains.

Figures (19)

• Figure 1: Framework for extracting and analyzing SDOH information from transplant evaluation notes. a) Schematic overview of liver transplant care journey. Decision outcomes shown in purple. b) Schematic overview of SDOH snapshot creation and analysis pipeline. Clinical notes are processed using LLMs to extract both (i) 23 SDOH dimensions describing patient circumstances* and (ii) clinical decisions/outcomes not captured in structured data (e.g., psychosocial risk assessments, transplant recommendations). These extracted elements are combined with structured clinical and demographic data from the EHR to create comprehensive patient snapshots at evaluation. The integrated data enables (i) comparison of SDOH factor prevalence across demographic groups, (ii) identification of transition points where specific factors impact care progression, and (iii) decomposition analysis of how SDOH patterns and clinical factors explain demographic differences in care access. This approach surfaces both individual-level circumstances and population-level patterns that can guide resource allocation and policy decisions. c) Accuracy of GPT-4-Turbo-128k vs. ground truth annotations (n=101) for 28 questions, including 23 SDOH-related dimensions. d) Demographic composition of the study cohort (n=3,704). e) Prevalence of key clinical outcomes, including psychosocial recommendation status (Rec) and liver transplant (LT) listing rates. *SDOH colored by related theme (yellow='Substance Use'; green='Social Support'; blue='Access', and red='Psychological')
• Figure 2: Analysis of demographic disparities in liver transplant listing rates. a) Baseline prevalence rates for psychosocial and substance use factors identified in clinical notes. b) Heat map showing statistically significant differences in factor prevalence across demographic groups (two-proportion z-tests, p $<$ 0.05, FDR-corrected), expressed as percentage point differences from baseline; colored boxes represent statistically significant differences from patient average; blue indicates lower rates, red indicates higher rates. c) Blinder-Oaxaca decomposition analysis quantifying explained and unexplained components of listing probability disparities, showing independent contributions of liver health metrics, SDOH features, and temporal effects.
• Figure 3: Demographic and SDOH variation across liver transplant outcomes. a) Percentage of patients reaching each evaluation milestone$^{\dag}$ stratified by demographic group, showing progression from initial psychosocial risk assessment through listing. Striped bars indicate significant differences from overall cohort means (FDR-corrected two-proportion z-tests). b) Heat map showing significant differences in SDOH factor prevalence between patients who did versus did not achieve each outcome (two-proportion z-tests, p $<$ 0.05, FDR-corrected); blue indicates higher rates, red indicates lower rates, blank cells indicate non-significant differences. c) OLS regression coefficients with LLM-derived, clinical, and demographic features. Significant coefficients marked (* p$<$0.05, ** p$<$0.01, *** p$<$0.001) and colored based on whether they have a positive (red) or negative (blue) impact on listing. $^{\dag}$Note on outcome classifications: "Recommended (Yes)" refers only to patients receiving unconditional recommendations, while "Recommended (Provisionally)" is a separate group. These two recommendation types are mutually exclusive. The "Overall" group includes both provisionally and unconditionally recommended patients. All other outcomes can co-occur.
• Figure 4: Model performance and feature analysis for psychosocial recommendation prediction. a) Comparison of average AUROC (w. 95% CI) across six combinations of clinical, demographic, and LLM-derived feature sets. Feature sets including LLM-derived features shown in blue. b) Confusion matrix for the LLM-SDOH + Clinical + Demographic combined feature model with normalized percentages over true values (rows). c) SHAP (SHapley Additive exPlanations) values for the top 15 features for the model with all feature sets.
• Figure 5: Model performance and feature analysis for liver transplant listing prediction. a) Comparison of average AUROC (w. 95% CI) across six combinations of clinical, demographic, and LLM-derived feature sets. Feature sets including LLM-derived features shown in blue. b) Confusion matrix for the Clinical (left) and Clinical + LLM-SDOH combined feature model (right) with normalized percentages over true values (rows). c) SHAP (SHapley Additive exPlanations) values for the top 15 features for the model with all feature sets.