Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuResonance: Exploring Feedback Experiences for Fostering the Inter-brain Synchronization

Jamie Ngoc Dinh, Snehesh Shrestha, You-Jin Kim, Jun Nishida, Myungin Lee

TL;DR

NeuResonance investigates whether external feedback in visual, auditory, and haptic modalities can boost Inter-brain Synchronization (IBS) during collaborative tasks. The authors deploy a real-time, dual-EEG hyperscanning pipeline using Emotiv EPOC X headsets, motion capture, and CCorr-based IBS estimation to drive feedback delivered via on-body projection, harmonic-note audio shifts, and vibrotactile bands. Results show that auditory and haptic feedback significantly enhance IBS relative to no feedback, while visual feedback via on-body projection provides limited gains, enabling formulation of design guidelines for IBS augmentation. The work offers actionable insights for applications in physical therapy and collaborative problem solving, highlighting modality-specific trade-offs and the potential for personalized feedback strategies to maximize IBS and teamwork outcomes.

Abstract

When several individuals collaborate on a shared task, their brain activities often synchronize. This phenomenon, known as Inter-brain Synchronization (IBS), is notable for inducing prosocial outcomes such as enhanced interpersonal feelings, including closeness, trust, empathy, and more. Further strengthening the IBS with the aid of external feedback would be beneficial for scenarios where those prosocial feelings play a vital role in interpersonal communication, such as rehabilitation between a therapist and a patient, motor skill learning between a teacher and a student, and group performance art. This paper investigates whether visual, auditory, and haptic feedback of the IBS level can further enhance its intensity, offering design recommendations for feedback systems in IBS. We report findings when three different types of feedback were provided: IBS level feedback by means of on-body projection mapping, sonification using chords, and vibration bands attached to the wrist.

NeuResonance: Exploring Feedback Experiences for Fostering the Inter-brain Synchronization

TL;DR

NeuResonance investigates whether external feedback in visual, auditory, and haptic modalities can boost Inter-brain Synchronization (IBS) during collaborative tasks. The authors deploy a real-time, dual-EEG hyperscanning pipeline using Emotiv EPOC X headsets, motion capture, and CCorr-based IBS estimation to drive feedback delivered via on-body projection, harmonic-note audio shifts, and vibrotactile bands. Results show that auditory and haptic feedback significantly enhance IBS relative to no feedback, while visual feedback via on-body projection provides limited gains, enabling formulation of design guidelines for IBS augmentation. The work offers actionable insights for applications in physical therapy and collaborative problem solving, highlighting modality-specific trade-offs and the potential for personalized feedback strategies to maximize IBS and teamwork outcomes.

Abstract

When several individuals collaborate on a shared task, their brain activities often synchronize. This phenomenon, known as Inter-brain Synchronization (IBS), is notable for inducing prosocial outcomes such as enhanced interpersonal feelings, including closeness, trust, empathy, and more. Further strengthening the IBS with the aid of external feedback would be beneficial for scenarios where those prosocial feelings play a vital role in interpersonal communication, such as rehabilitation between a therapist and a patient, motor skill learning between a teacher and a student, and group performance art. This paper investigates whether visual, auditory, and haptic feedback of the IBS level can further enhance its intensity, offering design recommendations for feedback systems in IBS. We report findings when three different types of feedback were provided: IBS level feedback by means of on-body projection mapping, sonification using chords, and vibration bands attached to the wrist.

Paper Structure

This paper contains 61 sections, 2 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Hypothesis
  3. Inter-brain Sync and Joint Task
  4. External Stimuli and Brainwave
  5. Interactive Feedback Systems
  6. Visual Feedback
  7. Auditory Feedback
  8. Haptic Feedback
  9. Team Performance and Communication
  10. Our Hypothesis and Approach
  11. Implementation
  12. Feedback Design
  13. Visual Feedback
  14. Hypothesis H1 Rationale
  15. Feedback Design
  16. ...and 46 more sections

Figures (7)

  • Figure 1: Design approaches for the (a) Visual, (b) Auditory, and (c) Haptic feedback
  • Figure 2: (a) Emotiv EPOC X EEG headset with mocap markers (b) Block diagram of the proposed feedback system
  • Figure 3: (a) Motion artifacts in EEG recordings: No Motion (gray), Slow/Small Motion (blue), and Large/Fast Motion (red). (b) Top: Motion over time with rejected segments highlighted: linear motion (green) and angular velocity (orange). Bottom: EEG channel response with motion artifacts.
  • Figure 4: Different levels of IBS are illustrated by a ring-shaped visual representation. The amplitude of the waveform corresponds to the IBS intensity, where the sync levels are: (1) the lowest, (2) low, (3) high, and (4) the highest.
  • Figure 5: (a) User study setup (b) Hand Mimicry session (c) Cup & String session
  • ...and 2 more figures