Analyzing Cancer Patients' Experiences with Embedding-based Topic Modeling and LLMs

TL;DR

This study investigates neural topic modeling of cancer patient narratives using BERTopic and Top2Vec, with topic labeling via GPT-4o and subsequent evaluation against human judgments. BERTopic, especially when paired with clinical-domain embeddings such as BioClinicalBERT and sentence-based chunking, yielded more coherent and clinically interpretable topics than Top2Vec, enabling more precise insights into patient experiences. A global analysis across all 13 interviews using domain-specific embeddings revealed recurring themes in care coordination, decision-making, and emotional support, suggesting a path toward clinician-facing feedback tools that surface patient voices efficiently. The work highlights the potential and limitations of multilingual clinical NLP for transforming unstructured patient narratives into actionable healthcare improvements, while outlining future directions toward multilingual data, expert validation, and integration into clinical workflows.

Abstract

This study investigates the use of neural topic modeling and LLMs to uncover meaningful themes from patient storytelling data, to offer insights that could contribute to more patient-oriented healthcare practices. We analyze a collection of transcribed interviews with cancer patients (132,722 words in 13 interviews). We first evaluate BERTopic and Top2Vec for individual interview summarization by using similar preprocessing, chunking, and clustering configurations to ensure a fair comparison on Keyword Extraction. LLMs (GPT4) are then used for the next step topic labeling. Their outputs for a single interview (I0) are rated through a small-scale human evaluation, focusing on {coherence}, {clarity}, and {relevance}. Based on the preliminary results and evaluation, BERTopic shows stronger performance and is selected for further experimentation using three {clinically oriented embedding} models. We then analyzed the full interview collection with the best model setting. Results show that domain-specific embeddings improved topic \textit{precision} and \textit{interpretability}, with BioClinicalBERT producing the most consistent results across transcripts. The global analysis of the full dataset of 13 interviews, using the BioClinicalBERT embedding model, reveals the most dominant topics throughout all 13 interviews, namely ``Coordination and Communication in Cancer Care Management" and ``Patient Decision-Making in Cancer Treatment Journey''. Although the interviews are machine translations from Dutch to English, and clinical professionals are not involved in this evaluation, the findings suggest that neural topic modeling, particularly BERTopic, can help provide useful feedback to clinicians from patient interviews. This pipeline could support more efficient document navigation and strengthen the role of patients' voices in healthcare workflows.

Figures (6)

• Figure 1: Snippets of the I0 interview in Dutch (left image) and English translation (right image)
• Figure 2: Framework Design of Overall Experimental Investigation
• Figure 3: Distribution of ratings for Top2Vec and BERTopic topics (Q1), averaged over 3 raters. Note that Top2Vec has generated 19 topics and BERTopic has generated 17 topics.
• Figure 4: Distribution of ratings for Top2Vec and BERTopic keywords (Q3), averaged over 3 raters. Note that Top2Vec has generated 19 topics and BERTopic has generated 17 topics.
• Figure 5: BERT-topic Deployment on all Interviews with Clinical LM Embeddings