Facilitating Mixed-Methods Analysis with Computational Notebooks

TL;DR

The paper tackles the challenge of conducting mixed-methods research within a single, notebook-based environment by proposing a three-component design (Data Extraction & Preparation, Data Summary & Aggregation, Data Integration & Interpretation) that aligns with Pirolli and Card's sensemaking framework. It describes a prototype-driven design with side-by-side widgets sharing a common variable space, enabling foraging, summarization, and synthesis of qualitative and quantitative data in one workspace. A scenario study with three HCI researchers provides initial qualitative feedback on usability, requirements, and potential pain points, highlighting needs such as data granularity control, data provenance, synchronization, scalability, and generalizability. The authors aim to implement a working system and conduct further user studies, with future work including automation suggestions to streamline the workflow. The work advances mixed-methods research by reducing context switching and proposing a unified, interactive notebook environment for integrated data exploration and interpretation.

Abstract

Data exploration is an important aspect of the workflow of mixed-methods researchers, who conduct both qualitative and quantitative analysis. However, there currently exists few tools that adequately support both types of analysis simultaneously, forcing researchers to context-switch between different tools and increasing their mental burden when integrating the results. To address this gap, we propose a unified environment that facilitates mixed-methods analysis in a computational notebook-based settings. We conduct a scenario study with three HCI mixed-methods researchers to gather feedback on our design concept and to understand our users' needs and requirements.