Bridging Human Interpretation and Machine Representation: A Landscape of Qualitative Data Analysis in the LLM Era
Xinyu Pi, Qisen Yang, Chuong Nguyen, Hua Shen
TL;DR
This work introduces a 4×4 landscape that separates meaning-making (M1–M4) from modeling (D1–D4) to classify qualitative outputs and guide LLM-assisted QR. By annotating hundreds of QR papers and validating with a GPT-based pass, the authors reveal a systemic bias toward shallow meaning and low-commitment modeling in current automation, underscoring the need for explicit interpretive commitments and dynamic modeling. The paper then outlines governance, evaluation, and infrastructure requirements to enable reliable, auditable QR in the LLM era, including shared codebooks, traceability, and fitness-to-corpus metrics. It concludes with an agenda for researchers to push QR outputs toward deeper, theoretically informed, and dynamically evolving representations while acknowledging practical risks and ethical safeguards.
Abstract
LLMs are increasingly used to support qualitative research, yet existing systems produce outputs that vary widely--from trace-faithful summaries to theory-mediated explanations and system models. To make these differences explicit, we introduce a 4$\times$4 landscape crossing four levels of meaning-making (descriptive, categorical, interpretive, theoretical) with four levels of modeling (static structure, stages/timelines, causal pathways, feedback dynamics). Applying the landscape to prior LLM-based automation highlights a strong skew toward low-level meaning and low-commitment representations, with few reliable attempts at interpretive/theoretical inference or dynamical modeling. Based on the revealed gap, we outline an agenda for applying and building LLM-systems that make their interpretive and modeling commitments explicit, selectable, and governable.