Misfitting With AI: How Blind People Verify and Contest AI Errors

TL;DR

The paper investigates how blind users verify AI VAT outputs and contest their errors, addressing a gap where most XAI research centers sighted experiences. Through qualitative interviews with 26 blind or low-vision individuals, the study identifies processing and cross-cultural errors in AI VAT and documents a repertoire of verification strategies—ranging from everyday experimentation and sensory checks to involving sighted helpers and cross-device triangulation. Framed by the misfitting/fitting lens from feminist disability studies, the work shows that AI VAT often misfits blind users, especially those with marginalized identities, and advocates for disability-centered audits and more accessible, contestable explanations beyond standard confidence scores. The findings yield practical implications for responsible AI, including interactive camera guidance, inclusive feedback mechanisms, and oppression-aware design that centers disabled ways of knowing to improve trustworthy visual access for all users.

Abstract

Blind people use artificial intelligence-enabled visual assistance technologies (AI VAT) to gain visual access in their everyday lives, but these technologies are embedded with errors that may be difficult to verify non-visually. Previous studies have primarily explored sighted users' understanding of AI output and created vision-dependent explainable AI (XAI) features. We extend this body of literature by conducting an in-depth qualitative study with 26 blind people to understand their verification experiences and preferences. We begin by describing errors blind people encounter, highlighting how AI VAT fails to support complex document layouts, diverse languages, and cultural artifacts. We then illuminate how blind people make sense of AI through experimenting with AI VAT, employing non-visual skills, strategically including sighted people, and cross-referencing with other devices. Participants provided detailed opportunities for designing accessible XAI, such as affordances to support contestation. Informed by disability studies framework of misfitting and fitting, we unpacked harmful assumptions with AI VAT, underscoring the importance of celebrating disabled ways of knowing. Lastly, we offer practical takeaways for Responsible AI practice to push the field of accessible XAI forward.