Atlas of AI Risks: Enhancing Public Understanding of AI Risks

TL;DR

The paper tackles the problem that existing AI risk visualizations mostly emphasize technical issues and miss broader societal impacts, making risk understanding difficult for non-experts. It presents the Atlas of AI Risks, a narrative, visualization-based tool developed through crowdsourced design requirements and an LLM/GIM-backed content-generation pipeline, validated against a baseline in a US-representative study and demonstrated to generalize to 379 uses drawn from the AI Incident Database. Key contributions include six design requirements for ordinary-user–oriented risk visualization, a two-stage content-generation workflow, a map atlas interface with progressive disclosure, and evidence of improved usability, balance, and engagement. The work offers a practical, scalable approach for public deliberation, regulatory discussions, and education, with potential integration into AI city registers, consumer databases, and educational contexts.

Abstract

The prevailing methodologies for visualizing AI risks have focused on technical issues such as data biases and model inaccuracies, often overlooking broader societal risks like job loss and surveillance. Moreover, these visualizations are typically designed for tech-savvy individuals, neglecting those with limited technical skills. To address these challenges, we propose the Atlas of AI Risks-a narrative-style tool designed to map the broad risks associated with various AI technologies in a way that is understandable to non-technical individuals as well. To both develop and evaluate this tool, we conducted two crowdsourcing studies. The first, involving 40 participants, identified the design requirements for visualizing AI risks for decision-making and guided the development of the Atlas. The second study, with 140 participants reflecting the US population in terms of age, sex, and ethnicity, assessed the usability and aesthetics of the Atlas to ensure it met those requirements. Using facial recognition technology as a case study, we found that the Atlas is more user-friendly than a baseline visualization, with a more classic and expressive aesthetic, and is more effective in presenting a balanced assessment of the risks and benefits of facial recognition. Finally, we discuss how our design choices make the Atlas adaptable for broader use, allowing it to generalize across the diverse range of technology applications represented in a database that reports various AI incidents.

Figures (13)

• Figure 1: Proposing a tool for mapping risks of AI technology uses for ordinary individuals involved four steps. First, we conducted a crowdsourcing formative study to identify six design requirements for visualizing them. Next, using facial recognition as a case study, we asked participants of the formative study to generate its uses (including benefits, risks and mitigations) and evaluated them for correctness. Due to the low number of identified uses, we generated new uses by prompting the LLM and also evaluated them for correctness. Then, with the dataset in hand, we used information visualization techniques to build an interactive tool. We evaluated it against design requirements and its support for a decision-making task. Finally, to demonstrate how it generalizes, we visualized over 300 uses sourced from the AI Incident Database mcgregor2021preventing.
• Figure 2: The interface of the Atlas of AI Risks meets six design requirements: mapping many uses of technology (R1), presenting a balanced assessment of their risks and benefits (R2) categorizing them for better understanding (R3), reducing their complexity (R4), making them relevant to ordinary individuals (R5), and making their exploration engaging (R6).
• Figure 3: The final dashboard for use exploration. It includes an impact assessment card available in two versions—a brief tooltip (a) and a detailed profile (b) listing risks, benefits, and mitigations—as well as interactions for use browsing, onboarding, and exploration tracking (c-g).
• Figure 4: The Atlas outperformed baseline across all quantitative metrics. It offered a more balanced assessment of uses, scored higher in usability, visual aesthetics, and encouraged longer exploration time.
• Figure 5: Uses generated by the participants of the formative study through writing emails to regulators.