Reflective Motion and a Physical Canvas: Exploring Embodied Journaling in Virtual Reality

TL;DR

This paper investigates embodied journaling in VR, using the body and voice as a medium to reflect on lived experiences, challenging the primacy of linguistic expression. It introduces a prototype in Unity with a minimal VR space, a mirror, and pose estimation pipelines to capture motion and speech for later replay. In a within-subject study with $N=20$, embodied journaling yielded abstract motion representations that participants interpreted post hoc, whereas written journaling produced precise narrative descriptions. The results reveal complementary temporalities of reflection and suggest design opportunities to support non-linguistic self-understanding, with implications for somaesthetic HCI and reflective VR.

Abstract

In traditional journaling practices, authors express and process their thoughts by writing them down. We propose a somaesthetic-inspired alternative that uses the human body, rather than written words, as the medium of expression. We coin this embodied journaling, as people's isolated body movements and spoken words become the canvas of reflection. We implemented embodied journaling in virtual reality and conducted a within-subject user study (n=20) to explore the emergent behaviours from the process and to compare its expressive and reflective qualities to those of written journaling. When writing-based norms and affordances were absent, we found that participants defaulted towards unfiltered emotional expression, often forgoing words altogether. Rather, subconscious body motion and paralinguistic acoustic qualities unveiled deeper, sometimes hidden feelings, prompting reflection that happens after emotional expression rather than during it. We discuss both the capabilities and pitfalls of embodied journaling, ultimately challenging the idea that reflection culminates in linguistic reasoning.

Figures (5)

• Figure 1: This hypothetical storyboard was inspired by a typical journey of reflection during embodied journaling as most commonly expressed by participants. It depicts typical body poses chosen in different stages of reflection during the process of embodied journaling. Each body pose depicted here was chosen by a participant of our study. The figure shows both the first-person view in VR (having a mirror) and the body poses as seen from outside. In (a), the user adopts an arms-crossed pose while talking about a difficult experience. In (b), the user sits and looks up, pondering the negative experience while looking around the virtual environment. In (c), the user looks down at their feet, and the participant reported feeling "down" at that moment. In (d), the user walks around, turns around, and looks up, gaining some clarity after thinking.
• Figure 2: The full pose estimation pipeline for our embodied journaling system. In Session 1, we use existing tracking tools such as SDK tracking, along with synchronous algorithms such as VRIK and a Mediapipe fallback when the hands are out of view, to render the avatar pose in real-time. During session 1, we also collect movement data, processed offline using WHAM to generate more accurate animations. In Session 2, we use these reconstructed avatars to accurately replicate the initial journal.
• Figure 3: A visualization of the study procedure across the two sessions.
• Figure 4: Still images taken of participants' embodied journals that demonstrate some shared patterns of motion. In (a) and (b), participants sat down; in (c) and (d), participants made punching gestures; in (e) and (f), participants gazed downwards and closed off their posture; in (g) and (h), participants paced around the room. See our video figure for more examples.
• Figure 5: Violin plots for each of the metrics of the R2T2 scale loerakkerTechnologyWhichMakes2024b, with line markers for the max, mean, and min.