A Contextual Inquiry of People with Vision Impairments in Cooking

TL;DR

This work addresses how visually impaired individuals obtain and use contextual information about kitchen objects during cooking. It employs a three-phase contextual inquiry with 12 participants in their own kitchens to derive a taxonomy of object-context needs and the processes used to gather them, complemented by deployment considerations for AI-powered assistive tech. The findings reveal five primary and three secondary informational categories, multi-sensory information gathering, object modification strategies, and design guidelines for information granularity, modality, and form factors. The results offer a principled basis for developing context-aware, multimodal kitchen assistants that enhance independence and safety for visually impaired cooks. The study also discusses future directions around smart objects, background tracking, and user-centered deployment in real-world kitchens.

Abstract

Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies.

Figures (7)

• Figure 1: Contextual Inquiry: sample views of participants' home kitchens with varying spatial layouts (from wide to narrow), as well as object density.
• Figure 2: Contextual inquiry settings. Three camera views captured during the study to show the overall kitchen environment, ego-centric view, and a 3rd-person camera view of the person cooking. The top image is the overall still image of the kitchen setting. The bottom left image is the view from the stationary camera that shows our participant making a meal. The bottom right image is the egocentric view captured from the chest-mounted camera.
• Figure 3: Dishes made by our participants during the contextual inquiry in their kitchens.
• Figure 4: Some visual information can have health implications. Left: Sliced tomatoes still have stickers attached (highlighted); Right: Some vegetables were overlooked and left on the cutting board (highlighted).
• Figure 5: P9 using his left hand to touch the side of the wok to gauge the temperature of the wok.