Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ADx3: A Collaborative Workflow for High-Quality Accessible Audio Description

Lana Do, Shasta Ihorn, Charity Pitcher-Cooper, Juvenal Francisco Barajas, Gio Jung, Xuan Duy Anh Nguyen, Sanjay Mirani, Ilmi Yoon

TL;DR

ADx3 addresses the resource-intensive nature of high-quality audio description (AD) for blind and low-vision (BLV) audiences by integrating three modules: GenAD (vision-language model-based baseline generation guided by accessibility-promoting prompts), RefineAD (inclusive, HITL editing interface), and AdaptAD (on-demand user queries). Through expert evaluation of multiple modern VLM baselines, the paper demonstrates that while AI can produce fluent, contextually grounded drafts, high-quality AD requires targeted human refinement and user-driven adaptation. The work also presents a cohesive system architecture, a formative expert study, and demonstrations of how iterative feedback from edits and on-demand interactions can reveal and fill gaps in AI-generated and human-authored descriptions. Overall, ADx3 offers a scalable, collaborative workflow that can progressively improve AD quality and personalization, with future work focused on broader BLV participation, larger datasets, and more expressive synthetic speech to enhance engagement and accessibility.

Abstract

Audio description (AD) makes video content accessible to blind and low-vision (BLV) audiences, but producing high-quality descriptions is resource-intensive. Automated AD offers scalability, and prior studies show human-in-the-loop editing and user queries effectively improve narration. We introduce ADx3, a novel framework integrating these three modules: GenAD, upgrading baseline description generation with modern vision-language models (VLMs) guided by accessibility-informed prompting; RefineAD, supporting BLV and sighted users to view and edit drafts through an inclusive interface; and AdaptAD, enabling on-demand user queries. We evaluated GenAD in a study where seven accessibility specialists reviewed VLM-generated descriptions using professional guidelines. Findings show that with tailored prompting, VLMs produce good descriptions meeting basic standards, but excellent descriptions require human edits (RefineAD) and interaction (AdaptAD). ADx3 demonstrates collaborative workflows for accessible content creation, where components reinforce one another and enable continuous improvement: edits guide future baselines and user queries reveal gaps in AI-generated and human-authored descriptions.

ADx3: A Collaborative Workflow for High-Quality Accessible Audio Description

TL;DR

ADx3 addresses the resource-intensive nature of high-quality audio description (AD) for blind and low-vision (BLV) audiences by integrating three modules: GenAD (vision-language model-based baseline generation guided by accessibility-promoting prompts), RefineAD (inclusive, HITL editing interface), and AdaptAD (on-demand user queries). Through expert evaluation of multiple modern VLM baselines, the paper demonstrates that while AI can produce fluent, contextually grounded drafts, high-quality AD requires targeted human refinement and user-driven adaptation. The work also presents a cohesive system architecture, a formative expert study, and demonstrations of how iterative feedback from edits and on-demand interactions can reveal and fill gaps in AI-generated and human-authored descriptions. Overall, ADx3 offers a scalable, collaborative workflow that can progressively improve AD quality and personalization, with future work focused on broader BLV participation, larger datasets, and more expressive synthetic speech to enhance engagement and accessibility.

Abstract

Audio description (AD) makes video content accessible to blind and low-vision (BLV) audiences, but producing high-quality descriptions is resource-intensive. Automated AD offers scalability, and prior studies show human-in-the-loop editing and user queries effectively improve narration. We introduce ADx3, a novel framework integrating these three modules: GenAD, upgrading baseline description generation with modern vision-language models (VLMs) guided by accessibility-informed prompting; RefineAD, supporting BLV and sighted users to view and edit drafts through an inclusive interface; and AdaptAD, enabling on-demand user queries. We evaluated GenAD in a study where seven accessibility specialists reviewed VLM-generated descriptions using professional guidelines. Findings show that with tailored prompting, VLMs produce good descriptions meeting basic standards, but excellent descriptions require human edits (RefineAD) and interaction (AdaptAD). ADx3 demonstrates collaborative workflows for accessible content creation, where components reinforce one another and enable continuous improvement: edits guide future baselines and user queries reveal gaps in AI-generated and human-authored descriptions.
Paper Structure (43 sections, 22 figures, 2 tables)

This paper contains 43 sections, 22 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Accessibility Guidelines and Volunteer-driven Platforms for Audio Description
  4. Automating AD and Human-in-the-Loop Refinement
  5. Interactivity and Human-in-the-Loop Systems
  6. System
  7. Baseline Description Generation: GenAD
  8. Video Input and Scene Segmentation
  9. Scene-level Description Generation
  10. Description Optimization
  11. Illustrative Improvements from Prompting Design
  12. Text to Speech Audio Insertion
  13. Human in the Loop (HITL) Workflow
  14. On-Demand Description: AdaptAD
  15. Instant Description Request
  16. ...and 28 more sections

Figures (22)

  • Figure 1: ADx3 system architecture showing three integrated modules: GenAD for baseline generation, RefineAD for human editing, and AdaptAD for user-driven interaction with synchronized audio description
  • Figure 2: Scene-level description generation using contextual prompting from audio transcripts and accumulated prior scene descriptions.
  • Figure 3: Examples of prompting and optimization improvements across different videos: (a) refined prompting with guidelines and context, (b) condensing verbose drafts into natural pauses, and (c) merging text and visual cues into fluent descriptions.
  • Figure 4: Users preview the AI-generated draft, with inline and extended AD tracks in timeline bar. Buttons allow rating, optional feedback, and (by default) enabling collaborative editing to open the editor.
  • Figure 5: Editors refine AD drafts with features such as text revision, delivery style switching, track alignment, timestamp editing with the nudging tool, and adding/removing tracks. Accessibility support (screen reader, high contrast, keyboard navigation) enables BLV users to participate directly.
  • ...and 17 more figures