VisGuardian: A Lightweight Group-based Privacy Control Technique For Front Camera Data From AR Glasses in Home Environments

TL;DR

VisGuardian tackles the privacy-utility tension of always-on AR glasses by introducing a group-based, content-aware privacy control that operates entirely on-device. It combines a three-attribute privacy taxonomy with an interaction metaphor that lets users select a single object to propagate settings across semantically or spatially related items, using occlusion-based sanitization. Technical evaluation shows real-time performance with a latency of $14.0~\mathrm{ms}$ and a detection accuracy of $mAP_{50}=0.6704$, while a user study demonstrates faster permission configuration and higher satisfaction versus baselines. The work demonstrates both feasibility and practical value for protecting private content in home environments, offering design guidance for scalable, user-friendly privacy controls in AI-enabled AR systems.

Abstract

Always-on sensing of AI applications on AR glasses makes traditional permission techniques ill-suited for context-dependent visual data, especially within home environments. The home presents a highly challenging privacy context due to the high density of sensitive objects, and the frequent presence of non-consenting family members, and the intimate nature of daily routines, making it a critical focus area for scalable privacy control mechanisms. Existing fine-grained controls, while offering nuanced choices, are inefficient for managing multiple private objects. We propose VisGuardian, a fine-grained content-based visual permission technique for AR glasses. VisGuardian features a group-based control mechanism that enables users to efficiently manage permissions for multiple private objects. VisGuardian detects objects using YOLO and adopts a pre-classified schema to group them. By selecting a single object, users can efficiently obscure groups of related objects based on criteria including privacy sensitivity, object category, or spatial proximity. A technical evaluation shows VisGuardian achieves mAP50 of 0.6704 with only 14.0 ms latency and a 1.7% increase in battery consumption per hour. Furthermore, a user study (N=24) comparing VisGuardian to slider-based and object-based baselines found it to be significantly faster for setting permissions and was preferred by users for its efficiency, effectiveness, and ease of use.

Figures (6)

• Figure 1: The system flow of , where the processed video would be sent to the app upon users' group-based control. checks for privacy-sensitive content before the app has access to a video stream.
• Figure 2: The home environment where we conducted the experiment.
• Figure 3: The illustration for different techniques.
• Figure 4: The illustration of the interfaces for the experiment, where (a) the user select the task from the corresponding scenario, (b) the user input voice and then click to proceed, (c) the user viewed the answer.
• Figure 5: The average number of clicks and permission control time of different techniques. Errorbar indicated one standard deviation.