Academic case reports lack diversity: Assessing the presence and diversity of sociodemographic and behavioral factors related to Post COVID-19 Condition

TL;DR

The results highlight the effectiveness of transformer-based NER in extracting SDOH information from case reports, however, the findings also expose critical gaps in the representation of marginalized groups within PCC-related academic case reports.

Abstract

Understanding the prevalence, disparities, and symptom variations of Post COVID-19 Condition (PCC) for vulnerable populations is crucial to improving care and addressing intersecting inequities. This study aims to develop a comprehensive framework for integrating social determinants of health (SDOH) into PCC research by leveraging NLP techniques to analyze disparities and variations in SDOH representation within PCC case reports. Following construction of a PCC Case Report Corpus, comprising over 7,000 case reports from the LitCOVID repository, a subset of 709 reports were annotated with 26 core SDOH-related entity types using pre-trained named entity recognition (NER) models, human review, and data augmentation to improve quality, diversity and representation of entity types. An NLP pipeline integrating NER, natural language inference (NLI), trigram and frequency analyses was developed to extract and analyze these entities. Both encoder-only transformer models and RNN-based models were assessed for the NER objective. Fine-tuned encoder-only BERT models outperformed traditional RNN-based models in generalizability to distinct sentence structures and greater class sparsity. Exploratory analysis revealed variability in entity richness, with prevalent entities like condition, age, and access to care, and underrepresentation of sensitive categories like race and housing status. Trigram analysis highlighted frequent co-occurrences among entities, including age, gender, and condition. The NLI objective (entailment and contradiction analysis) showed attributes like "Experienced violence or abuse" and "Has medical insurance" had high entailment rates (82.4%-80.3%), while attributes such as "Is female-identifying," "Is married," and "Has a terminal condition" exhibited high contradiction rates (70.8%-98.5%).

Figures (5)

• Figure 1: The PCC Case Report Corpus was developed from LitCOVID case reports (January 2020–October 2023) featuring post-COVID symptoms, using keyword-based searches and strict inclusion criteria. After filtering and processing, 7,172 case reports were curated, with 709 selected for NER model development: 99 in subset 1 (keyword-filtered for diversity), 402 in subset 2, and 208 in subset 3 (randomly selected). Annotation involved initial NER, human review, and data augmentation, balancing quality and efficiency to enhance the model's ability to capture PCC complexities.
• Figure 2: Entity frequencies (percentage) for (a) model training and (b) evaluation case reports before and after augmentation. The percentages are calculated out of the total number of tokens, which were 321,368 before augmentation and 693,476 after augmentation for model training and 290,759 before augmentation and 711,743 after augmentation for model evaluation.
• Figure 3: Entity extraction and distribution analysis from 7,172 case report sections processed by the fine-tuned BERT Base Uncased model. (a) Distribution of entity counts per case report. (b) Frequency of non-‘O’ entity labels, out of all non-‘O’ entities. (c) Trigram frequency visualization for the top 25 most frequent entity types, where the order of occurrence is considered.
• Figure 4: (a) Percent Entailment and Contradiction by NLI Statement with RegEx Matching Within Model-Predicted Label Dimensions. (b) Percent entailment and contradiction by NLI statement, with RegEx matching on the full dataset.
• Figure :