A Vision for AI-Driven Adaptation of Dynamic AR Content to Users and Environments
Julian Rasch, Florian Müller, Francesco Chiossi
TL;DR
AR content placement in real-world settings faces the challenge of aligning digital overlays with both user motion and changing environments. The authors envision an integrated AI-driven system that combines computer vision, reinforcement learning, and large language models to automatically allocate information between environmental projections and head-mounted displays. This approach aims to minimize cognitive load by prioritizing content relevance and surface siting (projections vs. HMDs) in a context-aware manner. If realized, the framework could enable more intuitive, adaptive AR experiences across indoor/outdoor, stationary/moving scenarios, advancing AR toward a primary interaction medium.
Abstract
Augmented Reality (AR) is transforming the way we interact with virtual information in the physical world. By overlaying digital content in real-world environments, AR enables new forms of immersive and engaging experiences. However, existing AR systems often struggle to effectively manage the many interactive possibilities that AR presents. This vision paper speculates on AI-driven approaches for adaptive AR content placement, dynamically adjusting to user movement and environmental changes. By leveraging machine learning methods, such a system would intelligently manage content distribution between AR projections integrated into the external environment and fixed static content, enabling seamless UI layout and potentially reducing users' cognitive load. By exploring the possibilities of AI-driven dynamic AR content placement, we aim to envision new opportunities for innovation and improvement in various industries, from urban navigation and workplace productivity to immersive learning and beyond. This paper outlines a vision for the development of more intuitive, engaging, and effective AI-powered AR experiences.