Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Development

TL;DR

This study investigates the feasibility and impact of high-level AI labels designed to communicate practical trade-offs like predictive performance versus resource efficiency. Through semi-structured interviews with 16 practitioners across domains and an inductive thematic analysis, the authors assess interest, benefits, limitations, and trust considerations surrounding AI labeling. They find broad interest and potential for labels to improve decision-making and cross-domain communication, while highlighting risks of oversimplification, the need for customization, and the importance of credible certifying authorities. The work argues for interactive, adaptable labeling frameworks and independent certification to maximize trust and sustainability in AI development, and it contributes actionable guidelines for standardizing labeling procedures. Overall, AI labels can bridge gaps between experts and non-experts, but their design must balance simplicity with depth and connect seamlessly to other reporting forms to support trustworthy AI deployment.

Abstract

As artificial intelligence (AI) becomes integral to economy and society, communication gaps between developers, users, and stakeholders hinder trust and informed decision-making. High-level AI labels, inspired by frameworks like EU energy labels, have been proposed to make the properties of AI models more transparent. Without requiring deep technical expertise, they can inform on the trade-off between predictive performance and resource efficiency. However, the practical benefits and limitations of AI labeling remain underexplored. This study evaluates AI labeling through qualitative interviews along four key research questions. Based on thematic analysis and inductive coding, we found a broad range of practitioners to be interested in AI labeling (RQ1). They see benefits for alleviating communication gaps and aiding non-expert decision-makers, however limitations, misunderstandings, and suggestions for improvement were also discussed (RQ2). Compared to other reporting formats, interviewees positively evaluated the reduced complexity of labels, increasing overall comprehensibility (RQ3). Trust was influenced most by usability and the credibility of the responsible labeling authority, with mixed preferences for self-certification versus third-party certification (RQ4). Our Insights highlight that AI labels pose a trade-off between simplicity and complexity, which could be resolved by developing customizable and interactive labeling frameworks to address diverse user needs. Transparent labeling of resource efficiency also nudged interviewee priorities towards paying more attention to sustainability aspects during AI development. This study validates AI labels as a valuable tool for enhancing trust and communication in AI, offering actionable guidelines for their refinement and standardization.

Figures (8)

• Figure 1: Prototype AI labels, as generated by STREPfischer_towards_2024 and shown during interviews.
• Figure 2: Overview of interviewees' daily work affairs and how they relate to AI (x-axes represents the number of code occurrences).
• Figure 3: Interviewee sentiment towards labels, based on the number of positive and critical comments.
• Figure 4: Performance requirements when generally discussing AI and when facing our labels.
• Figure 5: Forms of reporting that are used by interviewees.