A Framework For Discussing LLMs as Tools for Qualitative Analysis

TL;DR

This paper addresses the ambiguous role of Large Language Models in qualitative analysis by integrating philosophy of science and cognitive linguistics to build a practical framework. It articulates two guiding questions—whether the LLM proposes or refutes a qualitative model and whether a human checks the LLM's decision-making directly—and situates these within positivist and constructivist viewpoints. The authors identify 'surfac ing counter-examples for human review' as a particularly promising use-case that supports cross-paradigm collaboration and scalable qualitative review. They propose a workflow that leverages traditional sampling to bootstrap analysis and uses LLMs to surface counter-examples from larger datasets, while preserving human interpretation and oversight, thereby enabling productive, human-in-the-loop collaboration across diverse epistemologies.

Abstract

We review discourses about the philosophy of science in qualitative research and evidence from cognitive linguistics in order to ground a framework for discussing the use of Large Language Models (LLMs) to support the qualitative analysis process. This framework involves asking two key questions: "is the LLM proposing or refuting a qualitative model?" and "is the human researcher checking the LLM's decision-making directly?". We then discuss an implication of this framework: that using LLMs to surface counter-examples for human review represents a promising space for the adoption of LLMs into the qualitative research process. This space is promising because it is a site of overlap between researchers working from a variety of philosophical assumptions, enabling productive cross-paradigm collaboration on tools and practices.