Automated Code Fix Suggestions for Accessibility Issues in Mobile Apps

TL;DR

FixAlly tackles the gap between automated accessibility scanning and actionable code fixes in mobile apps by employing a plan-localize-fix pipeline powered by a multi-agent LLM. It localizes the impacted UI element, generates multiple plausible code fixes, and validates them through a patch-build-run cycle, demonstrated on 14 SwiftUI iOS apps with a 77% plausible-fix rate and a 69.4% developer acceptance in a user study. The approach provides developers with multiple fix strategies and accompanying rationale, emphasizing localizability in source code and design integrity. Overall, the work demonstrates the feasibility and practical value of LLM-driven automated accessibility repair for mobile apps, while outlining directions for cross-platform generalization and cross-file handling.

Abstract

Accessibility is crucial for inclusive app usability, yet developers often struggle to identify and fix app accessibility issues due to a lack of awareness, expertise, and inadequate tools. Current accessibility testing tools can identify accessibility issues but may not always provide guidance on how to address them. We introduce FixAlly, an automated tool designed to suggest source code fixes for accessibility issues detected by automated accessibility scanners. FixAlly employs a multi-agent LLM architecture to generate fix strategies, localize issues within the source code, and propose code modification suggestions to fix the accessibility issue. Our empirical study demonstrates FixAlly's capability in suggesting fixes that resolve issues found by accessibility scanners -- with an effectiveness of 77% in generating plausible fix suggestions -- and our survey of 12 iOS developers finds they would be willing to accept 69.4% of evaluated fix suggestions.

Figures (6)

• Figure 1: Implementation of a dropdown list in SwiftUI.
• Figure 2: Localization of the color contrast issue in the source code.
• Figure 3: FixAlly's approach, consisting of 1) Data Processing which instruments the application and navigates to various screens via GUI tests to capture accessibility scans and screenshots, 2) Suggestion Generation which uses a multi-agent LLM architecture to generate fix suggestions for each detected issue from the input screenshot, source code, and issue descriptions, and 3) Suggestion Assessment which captures a new accessibility scan of the patched app and GUI screen and compares it to the prior report to determine if the fix suggestion resolved the issue.
• Figure 4: Multi-level, hybrid localization architecture: Static analysis of the UI hierarchy identifies parent and descendant views. LLM-based rating evaluates the match of each individual code snippet to the screenshot. Finally, LLM-based comparison examines the highly matched views to determine the most likely code snippet for applying the fix.
• Figure 5: Three different types of issues that were fixed by FixAlly. (a) The tool addresses the issue of small hit target size by identifying a group of semantically related elements and merging them into an interactive container. (b) Shows three generated fix suggestions for Text Clipped issue. The first plan involves replacing a single element with a group of elements while preserving the functionality(c) FixAlly addresses the contrast issue of the "Dismiss" button, while also maintaining design integrity by applying the font change to another similar button.