Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context-aware Adaptive Visualizations for Critical Decision Making

Angela Lopez-Cardona, Mireia Masias Bruns, Nuwan T. Attygalle, Sebastian Idesis, Matteo Salvatori, Konstantinos Raftopoulos, Konstantinos Oikonomou, Saravanakumar Duraisamy, Parvin Emami, Nacera Latreche, Alaa Eddine Anis Sahraoui, Michalis Vakallelis, Jean Vanderdonckt, Ioannis Arapakis, Luis A. Leiva

TL;DR

This work tackles the lack of real-time, cognitively aware visualizations for critical decision-making by introducing Symbiotik, a neuroadaptive visualization framework that estimates mental workload from EEG and behavioral signals and dynamically adapts dashboards through reinforcement learning. The system architecture integrates an end-to-end Event-Condition-Action loop, a compact data representation of neural signals, a neural-mechanistic understanding module, layout-specific RL agents, and a scalable Infovis Gateway for real-time deployment. Key contributions include the NMU mapping of EEG bands to cognitive states, per-layout RL policies trained on real-world data, and an adaptation engine that translates policy decisions into actionable dashboard changes via a JSON config. A 120-participant study across three visualization types demonstrates improvements in task performance and engagement, supporting the approach’s scalability and potential for real-time, user-centered visualization in complex decision environments.

Abstract

Effective decision-making often relies on timely insights from complex visual data. While Information Visualization (InfoVis) dashboards can support this process, they rarely adapt to users' cognitive state, and less so in real time. We present Symbiotik, an intelligent, context-aware adaptive visualization system that leverages neurophysiological signals to estimate mental workload (MWL) and dynamically adapt visual dashboards using reinforcement learning (RL). Through a user study with 120 participants and three visualization types, we demonstrate that our approach improves task performance and engagement. Symbiotik offers a scalable, real-time adaptation architecture, and a validated methodology for neuroadaptive user interfaces.

Context-aware Adaptive Visualizations for Critical Decision Making

TL;DR

This work tackles the lack of real-time, cognitively aware visualizations for critical decision-making by introducing Symbiotik, a neuroadaptive visualization framework that estimates mental workload from EEG and behavioral signals and dynamically adapts dashboards through reinforcement learning. The system architecture integrates an end-to-end Event-Condition-Action loop, a compact data representation of neural signals, a neural-mechanistic understanding module, layout-specific RL agents, and a scalable Infovis Gateway for real-time deployment. Key contributions include the NMU mapping of EEG bands to cognitive states, per-layout RL policies trained on real-world data, and an adaptation engine that translates policy decisions into actionable dashboard changes via a JSON config. A 120-participant study across three visualization types demonstrates improvements in task performance and engagement, supporting the approach’s scalability and potential for real-time, user-centered visualization in complex decision environments.

Abstract

Effective decision-making often relies on timely insights from complex visual data. While Information Visualization (InfoVis) dashboards can support this process, they rarely adapt to users' cognitive state, and less so in real time. We present Symbiotik, an intelligent, context-aware adaptive visualization system that leverages neurophysiological signals to estimate mental workload (MWL) and dynamically adapt visual dashboards using reinforcement learning (RL). Through a user study with 120 participants and three visualization types, we demonstrate that our approach improves task performance and engagement. Symbiotik offers a scalable, real-time adaptation architecture, and a validated methodology for neuroadaptive user interfaces.

Paper Structure

This paper contains 11 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Use Case
  3. Data Collection
  4. System Description
  5. Physiological Sensing Module
  6. Compact Data Representation
  7. Neural-Mechanistic Understanding Module
  8. Reinforcement Learning Module
  9. Adaptation Engine
  10. Infovis Gateway
  11. Conclusions and Future Work

Figures (3)

  • Figure 1: Crime Investigation Dashboard.
  • Figure 2: Symbiotik high-level software architecture.
  • Figure 3: EEG preprocessing pipeline: (A) Raw signal recording; (B) Bandpass filtering; (C) FFT to convert to frequency domain; (D) Time-frequency analysis to compare every 2 seconds the power of the following frequency bands: (E) Delta, Theta, Alpha, and Beta.