Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults Explore and Learn Smartphone Applications

TL;DR

Aging populations face significant digital-literacy barriers as smartphone ecosystems evolve. The authors conduct a two-phase study—Phase 1 a workshop with 18 older adults to identify smartphone-app exploration challenges and AR intervention ideas, and Phase 2 tech-probe participatory design with 15 participants to co-create AR-supported learning tools, implemented on HoloLens 2. Findings show AR can reduce both physical and cognitive burdens during multi-app exploration and trial-and-error learning, while shaping concrete interaction-design guidelines (e.g., extended screens, terminology explanations, and state-saving previews). The work offers a path toward inclusive AR-enabled learning for the elderly, with implications for independent digital literacy and potential future integrations with healthcare contexts and collaborative learning scenarios.

Abstract

The global aging trend compels older adults to navigate the evolving digital landscape, presenting a substantial challenge in mastering smartphone applications. While Augmented Reality (AR) holds promise for enhancing learning and user experience, its role in aiding older adults' smartphone app exploration remains insufficiently explored. Therefore, we conducted a two-phase study: (1) a workshop with 18 older adults to identify app exploration challenges and potential AR interventions, and (2) tech-probe participatory design sessions with 15 participants to co-create AR support tools. Our research highlights AR's effectiveness in reducing physical and cognitive strain among older adults during app exploration, especially during multi-app usage and the trial-and-error learning process. We also examined their interactional experiences with AR, yielding design considerations on tailoring AR tools for smartphone app exploration. Ultimately, our study unveils the prospective landscape of AR in supporting the older demographic, both presently and in future scenarios.

Figures (8)

• Figure 1: The Overview of the Two-phase Study. In Phase 1, (a) shows the introduction of AR's capabilities to participants, (b) shows their experience using AR, (c) shows the discussion of smartphone app exploration challenges, and (d) encapsulates their collaborative ideation on using AR to tackle these obstacles. Based on the findings from Phase 1, we built a tech-probe for Phase 2 participatory design. (e) and (f) display participants engaging with our AR tutorial and probe respectively, while (g) captures the essence of our participatory design approach with a participant sketching her ideas.
• Figure 2: Features of the Design Probe. (a) enlarges and explains a terminology (in this example, it is "aesthetic medicine") and shows the enlarged explanation on the side of the smartphone. (b) highlights the hidden feature and explains it (in this example, it means "sliding" ) on the side of the smartphone. (c) uses a ghost hand animation to demonstrate the feature's meaning. (d) shows a 3D cat as an accompanying assistant. (e) demonstrates a vertical extended screen on the top of the smartphone. (f) demonstrates a horizontal extended screen on the side of the smartphone.
• Figure 3: Older Adults' Drawing Samples. (a) shows that they would like AR tools to display their operation flow and summarize previously filled-in information so that in the confirmation stage of a task, they can check the information more easily and confidently. (b) shows that they wanted AR to present three apps on the same screen for them to compare the prices of vegetables. (c) shows that they wanted AR to record and save their trial-and-error process so they could return to any saved state. (d) shows that they wanted a custom icon to record multi-app usage and show which states they were currently in.
• Figure 4: Participants' Vision of Employing AR to Augment Visual Interactions. (a) illustrates 3D words in an exaggerated brush style that seem to "jump out" from the display. (b) depicts a gesture-induced screen magnification. (c) visualizes a finger simulating a magnifying effect. (d) reveals a multi-tiered magnification process.
• Figure 5: Participants' Vision of Utilizing AR to Alleviate Cognitive Load. (a) illustrates a custom icon for multi-app tasks, organized by individual apps. (b) demonstrates the integration of diverse information from multiple apps onto a single display. (c) portrays the simultaneous presentation of three apps, designed for efficient price comparison. (d) highlights a reminder board that aids in refocusing the user's attention to their current task following interruptions.