Patients Speak, AI Listens: LLM-based Analysis of Online Reviews Uncovers Key Drivers for Urgent Care Satisfaction

TL;DR

The paper addresses how to understand urgent care patient satisfaction beyond traditional surveys by applying an LLM-based aspect-based sentiment analysis (ABSA) to Google Maps reviews from the DMV region and Florida. It collects a large dataset, defines five service aspects, validates ABSA with manual annotations, and assesses multiple LLMs before selecting GPT-4o mini for analysis. Regression analyses link per-hospital sentiment scores to overall ratings while controlling for Census Block Group socioeconomic factors, revealing that interpersonal factors and operational efficiency are the strongest, independent drivers of satisfaction, with technical quality, finances, and facilities contributing little once these are accounted for. The findings highlight the value of crowdsourced reviews and LLM-enabled ABSA for informing targeted improvements in urgent care and demonstrate robustness across sensitivity analyses and region-specific contexts. The work underscores the potential of scalable, data-driven insights to enhance patient experiences and guide policy and operational decisions in community health care.

Abstract

Investigating the public experience of urgent care facilities is essential for promoting community healthcare development. Traditional survey methods often fall short due to limited scope, time, and spatial coverage. Crowdsourcing through online reviews or social media offers a valuable approach to gaining such insights. With recent advancements in large language models (LLMs), extracting nuanced perceptions from reviews has become feasible. This study collects Google Maps reviews across the DMV and Florida areas and conducts prompt engineering with the GPT model to analyze the aspect-based sentiment of urgent care. We first analyze the geospatial patterns of various aspects, including interpersonal factors, operational efficiency, technical quality, finances, and facilities. Next, we determine Census Block Group(CBG)-level characteristics underpinning differences in public perception, including population density, median income, GINI Index, rent-to-income ratio, household below poverty rate, no insurance rate, and unemployment rate. Our results show that interpersonal factors and operational efficiency emerge as the strongest determinants of patient satisfaction in urgent care, while technical quality, finances, and facilities show no significant independent effects when adjusted for in multivariate models. Among socioeconomic and demographic factors, only population density demonstrates a significant but modest association with patient ratings, while the remaining factors exhibit no significant correlations. Overall, this study highlights the potential of crowdsourcing to uncover the key factors that matter to residents and provide valuable insights for stakeholders to improve public satisfaction with urgent care.

Figures (5)

• Figure 1: Workflow of data processing and analysis of Google Maps reviews. a. Distribution of urgent care facilities in Florida and DMV (the District of Columbia, Maryland, and Virginia). Each point represents an urgent care facility, with the point size indicating the volume of reviews. b. Examples of patient reviews, one negative and one positive. c. LLMs used to classify opinions extracted from Google Maps reviews.
• Figure 2: Performance on the sentiment classification of candidate models.
• Figure 3: Distribution of sentiment scores across five key aspects of patient experience by region.
• Figure 4: Geospatial distribution among health centers. a. Average Rating Score in DMV and FL. b. Sentiment Score among Aspects in DMV and FL.
• Figure 5: Distribution (diagonal cells) and pairwise association (lower triangle), and pairwise correlations between service aspects and ratings for DMV and Florida regions.