PD-Insighter: A Visual Analytics System to Monitor Daily Actions for Parkinson's Disease Treatment

TL;DR

PD-Insighter addresses a critical gap in understanding PD patients' motor function outside the clinic by coupling long-horizon visual analytics with context-rich immersive reconstructions. The system fuses an Overview Dashboard (action-driven, with heatmaps and distributions) and an Immersive Replay (AR-based body-environment reconstructions) to facilitate both broad pattern discovery and detailed investigation of specific events, using body variables such as trunk angle $ heta$, arm use $ ho^s_{ ext{arm}}$, foot position $ ho^s_{ ext{foot}}$, and weight shift $ ho^s_{ ext{weightShift}}$. A processing pipeline converts RGB/IMU data into pose estimates, action labels, and environment meshes, enabling tasks like identifying and filtering by action, discovering deficits, and contextualizing deficits within time and space. In a think-aloud study with six rehabilitation specialists, PD-Insighter enabled rapid insight into motion data across multiple levels of detail and supported contextual analysis with AR, suggesting potential to improve personalized therapy and remote monitoring for PD and potentially other motor disorders. The work provides design guidance for generalized multiperspective body motion analytics, balancing high-level longitudinal summaries with detailed, context-rich replays to inform clinical decision-making.

Abstract

People with Parkinson's Disease (PD) can slow the progression of their symptoms with physical therapy. However, clinicians lack insight into patients' motor function during daily life, preventing them from tailoring treatment protocols to patient needs. This paper introduces PD-Insighter, a system for comprehensive analysis of a person's daily movements for clinical review and decision-making. PD-Insighter provides an overview dashboard for discovering motor patterns and identifying critical deficits during activities of daily living and an immersive replay for closely studying the patient's body movements with environmental context. Developed using an iterative design study methodology in consultation with clinicians, we found that PD-Insighter's ability to aggregate and display data with respect to time, actions, and local environment enabled clinicians to assess a person's overall functioning during daily life outside the clinic. PD-Insighter's design offers future guidance for generalized multiperspective body motion analytics, which may significantly improve clinical decision-making and slow the functional decline of PD and other medical conditions.

Figures (10)

• Figure 2: Workflow Diagram: The pipeline and tools used to implement PD-Insighter. First, data collected about the person and their environment is processed to extract body pose, action labels, and an environment mesh. Next, PD-Insighter presents this data to clinicians through the Overview Dashboard, which visualizes the body variables and action labels together, and the Immersive Replay, which visualizes the body pose and environment mesh together.
• Figure 3: Data Collection: A person wearing IMUs acting out ADLs in our artificial apartment environment for data collection.
• Figure 4: Calculating Body Data: 1) Cardinal planes of motion, with the sagittal plane in blue and the coronal plane in red. 2) Skeleton rendered from human body pose coordinates, embedded with arrows encoding the body variables "arm use", "foot position", and "weight shift".
• Figure 5: An example use case of our Overview Dashboard: A clinician selects the Standing action (1) and Trunk and Arm Use body variables (2). In the Timeline window, PD-Insighter displays Standing events and Trunk and Arm Use temporal heatmaps (3a), with additional filters available for further investigation, and the High Trunk Angle filter selected (3b). Statistical metrics summarize the displayed data (4).
• Figure 6: Body Variable Representation: Moving the body variable legend slider changes the temporal heatmap to emphasize different motion thresholds (1). The distributions overlaid on the sliders can help clinicians quickly identify normal and abnormal distributions in weight shift balance and other body variables (2). The default temporal heatmap has more details and variations compared to the simplified heatmap that emphasizes outliers (3).