Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations

TL;DR

The paper reframes linting as a sociotechnical activity that extends from code to data visualizations, proposing crowd-powered evaluation via Community Notes as a form of human-computation linting. It develops an analytical framework (alignment chart and linting ladder) to categorize inputs and evaluation modes, and demonstrates how Community Notes can surface legitimate visual critiques while offering social insights. Key contributions include a conceptualization of sociotechnical lints, concrete examples from data visualizations, and a discussion of biases, uncertainty, and AI integration. The work highlights the potential and risks of crowd-driven linting for building trust and accountability in visual artifacts and suggests future research on hybrid human-AILinting systems and uncertainty communication.

Abstract

Traditionally, linters are code analysis tools that help developers by flagging potential issues from syntax and logic errors to enforcing syntactical and stylistic conventions. Recently, linting has been taken as an interface metaphor, allowing it to be extended to more complex inputs, such as visualizations, which demand a broader perspective and alternative approach to evaluation. We explore a further extended consideration of linting inputs, and modes of evaluation, across the puritanical, neutral, and rebellious dimensions. We specifically investigate the potential for leveraging human computation in linting operations through Community Notes -- crowd-sourced contextual text snippets aimed at checking and critiquing potentially accurate or misleading content on social media. We demonstrate that human-powered assessments not only identify misleading or error-prone visualizations but that integrating human computation enhances traditional linting by offering social insights. As is required these days, we consider the implications of building linters powered by Artificial Intelligence.

Figures (3)

• Figure 1: A characterization of the input space for linting as Euler diagram. Linters tend to exist in domains where the input space is expressive enough for there to be types of input that are not correct by default. These rich domains are common, for instance, language, text, visualization, film, and many other mediums support such expansive input. Within these spaces linters can only reason about a subset of possible input (which is itself only a subset of the space for which there are community norms), which is possibly intersecting the space of input the user wishes to make.
• Figure 2: Different systems organized in an alignment chart format kym_alignment_chart. This framing forgoes nuances between categories---such as human-machine teaming which might exist between evaluation neutral and rebel---and leaves consideration of those interactions to future work.
• Figure 3: Example of three Community Notes as applied to data visualizations.