CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development

TL;DR

This paper tackles the persistent problem of inaccessible web UI code by novice developers and evaluates the potential of AI coding assistants to improve accessibility by default. It presents CodeA11y, an accessibility aware GitHub Copilot extension that integrates three design goals by defaulting accessibility, identifying violations in real time, and reminding developers to complete manual steps. A formative study with 16 novices identifies three barriers to effective AI assisted accessibility, which informs the design of CodeA11y. A subsequent evaluation with 20 new participants shows that CodeA11y significantly improves accessibility outcomes for form labeling, button color contrast, and alt text, suggesting that AI copilots can be harnessed to promote accessibility in everyday development while also highlighting the need for ongoing developer education and better UI integration. The work demonstrates a concrete path toward embedding accessibility into AI coding tools and discusses broader implications for how such tools can influence developer behavior and nonfunctional requirements in practice.

Abstract

A persistent challenge in accessible computing is ensuring developers produce web UI code that supports assistive technologies. Despite numerous specialized accessibility tools, novice developers often remain unaware of them, leading to ~96% of web pages that contain accessibility violations. AI coding assistants, such as GitHub Copilot, could offer potential by generating accessibility-compliant code, but their impact remains uncertain. Our formative study with 16 developers without accessibility training revealed three key issues in AI-assisted coding: failure to prompt AI for accessibility, omitting crucial manual steps like replacing placeholder attributes, and the inability to verify compliance. To address these issues, we developed CodeA11y, a GitHub Copilot Extension, that suggests accessibility-compliant code and displays manual validation reminders. We evaluated it through a controlled study with another 20 novice developers. Our findings demonstrate its effectiveness in guiding novice developers by reinforcing accessibility practices throughout interactions, representing a significant step towards integrating accessibility into AI coding assistants.

Figures (5)

• Figure 1: Examples of task descriptions and visual references given to our participants: (a) Task 2 was to implement a new contact form for subscribing to a mailing list, and (b) Task 3 was to add a 'Top Stories' section with linked articles. Successfully completing them required proper labeling of the form elements and links, but this was not explicitly stated in the instructions.
• Figure 2: Mean Accessibility Evaluation Scores by Tasks and Copilot Usage: Higher scores indicate success.
• Figure 3: CodeA11y Architecture: Multi-agent workflow
• Figure 4: Contrasting responses for the same task across AI-assistants, showing differences in workflows. Developers had access to both the code and the rendered user interface.(a) and (d) represent conversations with the baseline assistant, and CodeA11y respectively. (b) and (e) show the buttons generated by each assistant in their default state. (c) and (f) display the buttons when hovered over, illustrating the differences in button color contrast.
• Figure 5: Mean Accessibility Evaluation Scores by Tasks and AI Assistant: Higher scores indicate success.