Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HealthPrism: A Visual Analytics System for Exploring Children's Physical and Mental Health Profiles with Multimodal Data

Zhihan Jiang, Handi Chen, Rui Zhou, Jing Deng, Xinchen Zhang, Running Zhao, Cong Xie, Yifang Wang, Edith C. H. Ngai

TL;DR

HealthPrism addresses how children's health is shaped by personal and family context using multimodal data from wearables and questionnaires. It introduces a gate-based multimodal learning model for health profiling and an interactive visual analytics system with Summary, Group, and Individual views to compare feature importance and influence across context and motion data. Quantitative evaluation shows the proposed health profiling model achieving the best mean AUC ($mAUC$) across settings, alongside case studies and expert interviews that demonstrate usability and insights. The work enables cross-modality interpretation, cohort-level screening, and efficient exploration of influential factors to inform targeted interventions.

Abstract

The correlation between children's personal and family characteristics (e.g., demographics and socioeconomic status) and their physical and mental health status has been extensively studied across various research domains, such as public health, medicine, and data science. Such studies can provide insights into the underlying factors affecting children's health and aid in the development of targeted interventions to improve their health outcomes. However, with the availability of multiple data sources, including context data (i.e., the background information of children) and motion data (i.e., sensor data measuring activities of children), new challenges have arisen due to the large-scale, heterogeneous, and multimodal nature of the data. Existing statistical hypothesis-based and learning model-based approaches have been inadequate for comprehensively analyzing the complex correlation between multimodal features and multi-dimensional health outcomes due to the limited information revealed. In this work, we first distill a set of design requirements from multiple levels through conducting a literature review and iteratively interviewing 11 experts from multiple domains (e.g., public health and medicine). Then, we propose HealthPrism, an interactive visual and analytics system for assisting researchers in exploring the importance and influence of various context and motion features on children's health status from multi-level perspectives. Within HealthPrism, a multimodal learning model with a gate mechanism is proposed for health profiling and cross-modality feature importance comparison. A set of visualization components is designed for experts to explore and understand multimodal data freely. We demonstrate the effectiveness and usability of HealthPrism through quantitative evaluation of the model performance, case studies, and expert interviews in associated domains.

HealthPrism: A Visual Analytics System for Exploring Children's Physical and Mental Health Profiles with Multimodal Data

TL;DR

HealthPrism addresses how children's health is shaped by personal and family context using multimodal data from wearables and questionnaires. It introduces a gate-based multimodal learning model for health profiling and an interactive visual analytics system with Summary, Group, and Individual views to compare feature importance and influence across context and motion data. Quantitative evaluation shows the proposed health profiling model achieving the best mean AUC () across settings, alongside case studies and expert interviews that demonstrate usability and insights. The work enables cross-modality interpretation, cohort-level screening, and efficient exploration of influential factors to inform targeted interventions.

Abstract

The correlation between children's personal and family characteristics (e.g., demographics and socioeconomic status) and their physical and mental health status has been extensively studied across various research domains, such as public health, medicine, and data science. Such studies can provide insights into the underlying factors affecting children's health and aid in the development of targeted interventions to improve their health outcomes. However, with the availability of multiple data sources, including context data (i.e., the background information of children) and motion data (i.e., sensor data measuring activities of children), new challenges have arisen due to the large-scale, heterogeneous, and multimodal nature of the data. Existing statistical hypothesis-based and learning model-based approaches have been inadequate for comprehensively analyzing the complex correlation between multimodal features and multi-dimensional health outcomes due to the limited information revealed. In this work, we first distill a set of design requirements from multiple levels through conducting a literature review and iteratively interviewing 11 experts from multiple domains (e.g., public health and medicine). Then, we propose HealthPrism, an interactive visual and analytics system for assisting researchers in exploring the importance and influence of various context and motion features on children's health status from multi-level perspectives. Within HealthPrism, a multimodal learning model with a gate mechanism is proposed for health profiling and cross-modality feature importance comparison. A set of visualization components is designed for experts to explore and understand multimodal data freely. We demonstrate the effectiveness and usability of HealthPrism through quantitative evaluation of the model performance, case studies, and expert interviews in associated domains.
Paper Structure (26 sections, 5 figures, 1 table)

This paper contains 26 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Visualization for Healthcare
  4. Model Interpretation and Visualization
  5. Background and System Overview
  6. Data Description and Pre-processing
  7. Design Process and Requirements
  8. System Overview
  9. Data Analysis
  10. Pattern Extraction
  11. Health Profiling Model
  12. Feature Importance Assessment
  13. Feature Influence Analysis
  14. Visual Design
  15. Summary View
  16. ...and 11 more sections

Figures (5)

  • Figure 1: Overview of the system and pipeline of HealthPrism.
  • Figure 2: An illustration of overall-level inspection. ①-③: Longer sleep duration on weekdays decreases physical functioning and psychosocial functioning but increases health confidence. ④-⑤: with the increasing family income, children's PHYF increases but CONN decreases.
  • Figure 3: An illustration of group-level inspection. ①-④: The Influence of academic performance and interest on female children's PHYF. ⑤: The influence of exercise habits of family members on male children's PHYF.
  • Figure 4: An illustration of individual-level inspection. ①-④: E3 selected an individual (ID 889) and found the learning mode an interesting feature. ⑤-⑨: E3 chose another individual (ID 436) within the same cluster of ID 889 to explore the influence of the learning mode. ④, ⑧, ①0: E3 further compared the context and motion features of these two individuals.
  • Figure 5: An illustration of health profile comparison between female and male children. ①-⑦: E8 adjusted the time window and found gender was important to CONN, PSYF, and RESI. ⑧: E8 further explore the feature importance of females and males. ⑨, ①0: Female's interest and performance on Chinese had different influences on their RESI. ①1: E8 used the interactive table to explore the correlation between the interest and performance of three major courses. ①2: E8 used the toggle switch to compare the motion features of females and males.