Medillustrator: Improving Retrospective Learning in Physicians' Continuous Medical Education via Multimodal Diagnostic Data Alignment and Representation

TL;DR

This work introduces Medillustrator, a visual analytics system crafted to facilitate novice physicians’ retrospective learning, which enables novice physicians to explore and review research cases at an overview level and analyze specific cases with consistent alignment of multimodal and reference data.

Abstract

Continuous Medical Education (CME) plays a vital role in physicians' ongoing professional development. Beyond immediate diagnoses, physicians utilize multimodal diagnostic data for retrospective learning, engaging in self-directed analysis and collaborative discussions with peers. However, learning from such data effectively poses challenges for novice physicians, including screening and identifying valuable research cases, achieving fine-grained alignment and representation of multimodal data at the semantic level, and conducting comprehensive contextual analysis aided by reference data. To tackle these challenges, we introduce Medillustrator, a visual analytics system crafted to facilitate novice physicians' retrospective learning. Our structured approach enables novice physicians to explore and review research cases at an overview level and analyze specific cases with consistent alignment of multimodal and reference data. Furthermore, physicians can record and review analyzed results to facilitate further retrospection. The efficacy of Medillustrator in enhancing physicians' retrospective learning processes is demonstrated through a comprehensive case study and a controlled in-lab between-subject user study.

Figures (6)

• Figure 1: Medillustrator is composed of a data process module, a modeling engine, and a visualization interface designed to align with the discovery-driven learning workflow.
• Figure 2: Glyph design in fusion modal and connections across fusion modal and unimodal. Current glyph design. - Alternatives based on the rose chart and box plot.
• Figure 3: Case Study Part I: Exploring high-value patient cases and addressing interpretation challenges. Choose the specialty and data time span. Identify "retreat" as one of the most frequent mentions. Enter "retreat" in the input box and initiate the search. Select the relevant glyph in the Embedding View. Notice the simultaneous appearance of three identical k-NNs in both the image and indicator modalities. Analyze the case in the Detail View, annotate findings with , and review the analysis using . Navigate to the neighboring case in the image modality and inspect it in the Detail View. Annotate findings with and review the diagnosis using , discovering an incorrect diagnosis. Save the results in the Record View by clicking .
• Figure 4: In Case Study Part II, the physicians' interaction workflow comprises: Filter for cases containing "herniation". Zoom in to view and discover two close cases (p1, p2) in Embedding View. Lasso the cases as a group to examine their distribution across modalities, finding the two cases are close in the image modality but distant in the other two modalities. Click on glyph of case p1 to examine it in Detail View. Examine and annotate analysis on the image. Check the aligned diagnosis on the image. Record the analysis of p1 in Record View. Click on glyph of case p2 to examine it in Detail View. Examine the image and annotate partial analysis in Detail View. Switch to check the contextual information between p1 and p2 in Information Exploration View. Observe Indicator subview and find that the lymphocyte percent of p2 exceed normal range. Review Medical History subview and find that p2 has a longer duration of illnesses than p1. Observe that p2 is examined abnormal in "sensation of right upper limb". Notice unusual signal values of C6-C7 area in Detail View, indicating potential lesions. Complete the analysis of p2 and verify its accuracy by checking the aligned diagnosis on the image. Record the analysis of p2 in Record View.
• Figure 5: Participants were randomly assigned to two groups: one using Medillustrator and the other using a baseline condition for case screening and analysis. The Medillustrator group received a tutorial and exploration session to become familiar with the system. Throughout the study, participants completed in-task surveys to evaluate system usability and effectiveness, followed by post-task interviews to gather their perceptions.