An Outlook into the Future of Egocentric Vision
Chiara Plizzari, Gabriele Goletto, Antonino Furnari, Siddhant Bansal, Francesco Ragusa, Giovanni Maria Farinella, Dima Damen, Tatiana Tommasi
TL;DR
This survey reframes egocentric vision by forecasting a future where wearables with outward cameras and multimodal overlays act as ego-centric assistants (EgoAI). It systematically maps envisioned everyday use-cases to core tasks—localisation, 3D scene understanding, recognition, anticipation, gaze, social behavior, pose, hand-object interactions, identification, summarisation, dialogue, and privacy—reviewing seminal works, current state-of-the-art methods, and relevant datasets. The analysis highlights gaps between present capabilities and the envisioned always-on, personalised EgoAI, emphasizing the need for multi-sensor integration, real-time operation, robust privacy-preserving approaches, and cross-task synergy. The paper concludes with concrete recommendations for immediate exploratory directions and stresses the value of large, diverse datasets (e.g., EPIC-KITCHENS, Ego4D) and emerging ego-language models to enable practical, human-centric egocentric assistance.
Abstract
What will the future be? We wonder! In this survey, we explore the gap between current research in egocentric vision and the ever-anticipated future, where wearable computing, with outward facing cameras and digital overlays, is expected to be integrated in our every day lives. To understand this gap, the article starts by envisaging the future through character-based stories, showcasing through examples the limitations of current technology. We then provide a mapping between this future and previously defined research tasks. For each task, we survey its seminal works, current state-of-the-art methodologies and available datasets, then reflect on shortcomings that limit its applicability to future research. Note that this survey focuses on software models for egocentric vision, independent of any specific hardware. The paper concludes with recommendations for areas of immediate explorations so as to unlock our path to the future always-on, personalised and life-enhancing egocentric vision.