How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI

TL;DR

The paper tackles the difficulty non-expert Disability Services Office professionals face when authoring alt text for complex science figures in HCI publications. It employs a two-study design using ChatGPT-4o to generate AI-assisted alt text for 12 DSOs and evaluates 11 HCI experts on the quality of outputs, with a rubric-based assessment and figure complexity analysis. Results show AI-assisted alt text generally improves quality and efficiency, though human oversight remains essential due to inaccuracies and prompting gaps, especially for complex figures. The findings support AI as a scaffolding tool to enhance accessibility workflows in academia and point to the need for training, standards, and careful integration into publishing practices.

Abstract

Disability Services Office (DSO) professionals at higher education institutions write alt text for {visual content}. However, due to the complexity of visual content, such as HCI figures in research publications, DSO professionals can struggle to write high-quality alt text if they lack subject expertise. Generative AI has shown potential in understanding figures and writing their descriptions, yet its support for DSO professionals is underexplored, and limited work evaluates the quality of alt text generated with AI assistance. In this work, we conducted two studies: first, we investigated generative AI support for writing alt text for HCI figures with 12 DSO professionals. Second, we recruited 11 HCI experts to evaluate the alt text written by DSO professionals. Findings show that alt text written solely by DSO professionals has lower quality than alt text written with AI assistance. AI assistance also helped DSO professionals write alt text more quickly and with greater confidence; however, they reported inefficiencies in interactions with the AI. Our work contributes to exploring AI support for non-subject expert accessibility professionals.

Figures (5)

• Figure 1: Overview of our analysis and two user studies. First, we select science figures, provide writing guidelines, and select an LLM model (i.e., ChatGPT-4o). Then, we conduct user studies with 12 DSO professionals, asking them to write alt text for science figures with and without AI assistance. We analyze participant feedback and the generated alt text for completeness and comprehensibility. We designed study 2 to evaluate the generated alt text by 11 HCI experts. We select a subgroup of 6 alt text figures each (2 alt text versions; one written with AI assistance and one without). We conducted an evaluation for alt text with two groups of HCI experts, where they saw each of six figures in randomized order. We seek feedback on alt text evaluation and their perception of text written by an AI or a human writer, or a combined human+AI.
• Figure 2: Participant confidence and output quality ratings for alt text authoring. Left: Participants rated their confidence in the alt text generated with AI-assistance higher than in self-generated alt text. Right: Participants with high confidence in HCI knowledge reported higher self-confidence (confidence in alt text produced without AI) as compared to their confidence in AI-assisted alt text.
• Figure 3: Participant interaction with ChatGPT for alt text writing. Left: Participant interaction ("write", "analyze", and "improve") count in descending order. P4 and P12 interactions were only captured on screen recording and not in chat history. Right: Participants mainly used ChatGPT to "write" alt text with direct commands, followed by "analyze" prompts to get LLMs contextualize or understand context themselves better. Only a few interactions were made to improve the produced alt text.
• Figure 4: Spider plot showing the rubric ratings for alt text written by two participants using the same method. The scores show the application of the rubric applied to the alt text in Table \ref{['tab-alt-text-filter-applied']} (refer to Table \ref{['tab-alt-text-rubric']} for detailed criteria (C1-C8).
• Figure 5: Box plot showing the rating scores for each method (LEFT: Human-Text, RIGHT: HAI-Text) by figure complexity (Simple, Moderately Complex, Complex). Participants rated HAI-Text higher in terms of quality compared to Human-Text, regardless of figure complexity. Overall, results indicate that participants rated HAI text better across all three figure complexity levels.