From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote Task Understanding

TL;DR

This work addresses the challenge that AR-guided tasks often promote procedural compliance over deep understanding. It introduces reflective prompts—Challenging Assumptions, Connections to Outcomes, and Hypothetical Scenarios—embedded in AR instructions and evaluates their impact via a formative study (N=9) and a within-subject evaluation (N=16) across two tasks (pour-over coffee and breadboard circuit assembly). Quantitative results show significant gains in objective understanding and information-seeking behavior (p<.05; $d=0.582$; increased keyword interactions with prompts, $p<.05$, $d=0.736$), while qualitative data reveal prompts are largely non-intrusive and trusted, though subjective understanding can decline in some contexts. The authors distill design guidelines for non-intrusive, adaptive, and autonomy-respecting reflective AR instructions and discuss broader applicability, limitations, and future work, including longitudinal studies and task diversification. Overall, this work demonstrates that integrating reflective elements into AR guidance can deepen user understanding without sacrificing performance, offering a practical path to more effective AR-based learning and task execution.

Abstract

Augmented Reality (AR) is a promising medium for guiding users through tasks, yet its impact on fostering deeper task understanding remains underexplored. This paper investigates the impact of reflective prompts -- strategic questions that encourage users to challenge assumptions, connect actions to outcomes, and consider hypothetical scenarios -- on task comprehension and performance. We conducted a two-phase study: a formative survey and co-design sessions (N=9) to develop reflective prompts, followed by a within-subject evaluation (N=16) comparing AR instructions with and without these prompts in coffee-making and circuit assembly tasks. Our results show that reflective prompts significantly improved objective task understanding and resulted in more proactive information acquisition behaviors during task completion. These findings highlight the potential of incorporating reflective elements into AR instructions to foster deeper engagement and learning. Based on data from both studies, we synthesized design guidelines for integrating reflective elements into AR systems to enhance user understanding without compromising task performance.

Figures (10)

• Figure 1: Formative study prototype system layout displaying one step of the Task A instructions (larger white text at the top), reflective prompts (small yellow text underneath) and ChatGPT's response (small white text at the bottom).
• Figure 2: As shown in Image (a), participants performed Task A wearing an Apple Vision Pro to follow instructions for making pour-over coffee. Equipment includes a kettle, coffee dripper, grinder, filters, and scale. As shown in Image (b), participants performed Task B wearing an Apple Vision Pro while assembling an electronic circuit using components and a breadboard.
• Figure 3: This figure summarizes reflection prompt types identified from the literature and evaluated for AR instruction following contexts. The left side presents five main categories of reflective prompts: Critical Reflection, Reflective Pauses, Metacognitive Awareness, Perspective Shifting, and Connection Identification, each with subtypes and example prompts. The right side shows results from co-design sessions (N=9), with colored boxes indicating the three subtypes deemed most effective and appropriate for AR instruction: Challenging Assumptions, Hypothetical Scenarios, and Connections to Outcomes.
• Figure 4: The three types of reflective prompts we concluded with the co-design session, with example prompts and the prompts we integrated into the system we used in the evaluation study for Task A and Task B respectively.
• Figure 5: Image (a), (b), (c), and (d) are screenshots of the participants' view in the Apple Vision Pro. Participants can click on words in the instructions to get more information wrapped in a rectangular window, as shown in (b) and (c). Interactable keywords are not distinguishable from regular instruction words. They can click on a "Previous" and a "Next" button to navigate through the instruction steps freely, as shown in (b). Reflective prompts are presented as non-interactable smaller yellow text, as shown in (b) and (d). English translations was added to the screenshots for clarity and are not part of the original interface.