MIND: Empowering Mental Health Clinicians with Multimodal Data Insights through a Narrative Dashboard

TL;DR

Mind addresses the challenge of integrating multimodal patient data in mental health by delivering data-driven narratives via a two-level dashboard. The authors design MIND through clinician co-design and implement a hybrid computation pipeline that combines rule-based data exploration with large language model-driven narration to generate concise, clinically grounded insights. In a mixed-method user study with 16 licensed clinicians, MIND outperformed a data-collection baseline on several quality and perception metrics (e.g., p<.001 for uncovering hidden insights; p=.004 for decision support), while preserving trust through verifiability. The work demonstrates the feasibility of narrative dashboards to enhance clinical reasoning with multimodal data and discusses avenues for real-world deployment and cross-domain applications.

Abstract

Advances in data collection enable the capture of rich patient-generated data: from passive sensing (e.g., wearables and smartphones) to active self-reports (e.g., cross-sectional surveys and ecological momentary assessments). Although prior research has demonstrated the utility of patient-generated data in mental healthcare, significant challenges remain in effectively presenting these data streams along with clinical data (e.g., clinical notes) for clinical decision-making. Through co-design sessions with five clinicians, we propose MIND, a large language model-powered dashboard designed to present clinically relevant multimodal data insights for mental healthcare. MIND presents multimodal insights through narrative text, complemented by charts communicating underlying data. Our user study (N=16) demonstrates that clinicians perceive MIND as a significant improvement over baseline methods, reporting improved performance to reveal hidden and clinically relevant data insights (p<.001) and support their decision-making (p=.004). Grounded in the study results, we discuss future research opportunities to integrate data narratives in broader clinical practices.

Figures (16)

• Figure 1: A comparison between the proposed narrative dashboard MIND (A) with a data collection dashboard design (B), showing the data from the same hypothetical patient. MIND presents computationally generated multimodal insights in a narrative way ❶, allowing clinicians to better understand multimodal patient data. Clinicians can also easily navigate through timeline ❷, filter by insight category ❸, and expand details on demand ❹.
• Figure 2: Illustration of the procedure of the co-design process.
• Figure 3: Four checkpoint designs (C1-C4) showing the co-design progression incorporating experts' ideas and feedback. The top row illustrates the design pattern, and the bottom row presents the prototype artifacts that instantiate each pattern. Information layout becomes more structured, while charts become more closely integrated with text content with each iteration. We present expanded views of the prototype artifacts for C1 (Fig. \ref{['fig:appendix:system-initial']}), C2 (Fig. \ref{['fig:appendix:system-pre-nar']}), and C3 (Fig. \ref{['fig:appendix:system-after-nar']}) in the Appendix.
• Figure 4: The L1 overview interface of MIND. MIND communicates multimodal data insights through a combination of linear and parallel storytelling techniques. On the outer level, MIND narrates insights using a linear four-part structure: Medical History (collapsed) ❶, Session Recap ❷, Patient Data Insights ❸, and Summary Today ❹, with overflowing content accessible through scrolling. Within the Patient Data Insights section, MIND further applies three parallel narrative threads based on the Biopsychosocial model for clinician-guided information faceting.
• Figure 5: Insight card as visual primitive. Three sections: insight description, insight types, and data source types, present the multimodal insights. The selector and drill-down buttons links the card with other parts of the MIND.