Accessible Text Descriptions for UpSet Plots

TL;DR

The paper tackles the accessibility barrier of UpSet plots for blind and low-vision users by developing an automatic, pattern-driven text description system. It starts with a survey of UpSet usage to identify describable patterns, then designs long and short descriptions grounded in semantic levels of context, implemented via a JSON-based standard and a web API. The authors validate the approach through BLV interviews and two contextual experiments with sighted users and LLM baselines, finding that text descriptions can be informative and, when paired with visuals, offer a curb-cut benefit to multiple user groups. The work contributes an open-source description generator, a scalable evaluation framework, and a pathway to extending text-based accessibility to other nonstandard chart types, advancing inclusive scientific communication.

Abstract

Data visualizations are typically not accessible to blind and low-vision (BLV) users. Automatically generating text descriptions offers an enticing mechanism for democratizing access to the information held in complex scientific charts, yet appropriate procedures for generating those texts remain elusive. Pursuing this issue, we study a single complex chart form: UpSet plots. UpSet Plots are a common way to analyze set data, an area largely unexplored by prior accessibility literature. By analyzing the patterns present in real-world examples, we develop a system for automatically captioning any UpSet plot. We evaluated the utility of our captions via semi-structured interviews with (N=11) BLV users and found that BLV users find them informative. In extensions, we find that sighted users can use our texts similarly to UpSet plots and that they are better than naive LLM usage.

Figures (13)

• Figure 1: An UpSet plot of the winners of major tennis tournaments, cropped to intersections with at least a size of 4. The sets are represented in the columns of the matrix; the set sizes as vertical bars; the size of the intersections as horizontal bars. Intersections are shown in the rows and identify the sets that intersect with filled in and connected circles. "Intersections" can be of only one set, here representing the players who have only won one tournament.
• Figure 2: Our short and long text descriptions annotated with Lundgard and Satyanarayan's lundgard_accessible_2022 semantic text model.
• Figure 3: Correct responses by condition (left) and by question (right). Each condition had 83 participants, all of whom saw all conditions.
• Figure 4: Example of the study interface for the Both condition. The title (right) is human-generated, the introduction leverages a human-provided specification. See https://submitterpaper.github.io/study/Upset-Alttext-User-Survey/0 for more.
• Figure 5: Percent of questions correctness by dataset and condition. Q6, which focuses on qualitative distribution similarity, had a high influence on the average correctness by dataset.