Augmented Journeys: Interactive Points of Interest for In-Car Augmented Reality

TL;DR

Augmented Journeys investigates in-car AR for passengers to interact with world-fixed POIs while traveling. It combines a survey (N=110), a pre-study (N=10), and a field study (N=21) to evaluate eye-gaze plus pinch interactions and three POI visualizations (Timeline, Minimap, List). The results show broad acceptance with a strong preference for the List visualization, while revealing notable hardware and motion-related limitations that affect 3D interaction and overall usability, prompting recommendations for multimodal input and lighter hardware. The work provides empirical guidance on POI interaction, visualization design, and integration into automotive contexts, with implications for scalable, passenger-centric AR interfaces in real-world driving scenarios.

Abstract

As passengers spend more time in vehicles, the demand for non-driving related tasks (NDRTs) increases. In-car Augmented Reality (AR) has the potential to enhance passenger experiences by enabling interaction with the environment through NDRTs using world-fixed Points of Interest (POIs). However, the effectiveness of existing interaction techniques and visualization methods for in-car AR remains unclear. Based on a survey (N=110) and a pre-study (N=10), we developed an interactive in-car AR system using a video see-through head-mounted display to engage with POIs via eye-gaze and pinch. Users could explore passed and upcoming POIs using three visualization techniques: List, Timeline, and Minimap. We evaluated the system's feasibility in a field study (N=21). Our findings indicate general acceptance of the system, with the List visualization being the preferred method for exploring POIs. Additionally, the study highlights limitations of current AR hardware, particularly the impact of vehicle movement on 3D interaction.

Figures (10)

• Figure 1: A Graph showing how and when passengers and drivers look for a missed point of interest. Multiple choice was possible.
• Figure 2: View of the pre-study. We use the Varjo XR-3 with additional optical tracking (left). POIs are visualized as spheres outside the vehicle (right). The POI with the red crosshair had to be selected via eye-gaze and a hardware button.
• Figure 3: The tracks used for the pre-study (blue) and the main-study (red). The studies were conducted in an industrial area with a 50km/h speed limit and moderate traffic. Traffic lights are annotated via icons.
• Figure 4: An overview of the UI elements used in the study. The Informations panel always correspondet to the currently selected POI. The List, Minimap, and Timeline were only used during their study-conditions respectively.
• Figure 5: Design of the world-fixed POIs with the location name, star rating, and representative image.