PLUTO: A Public Value Assessment Tool

TL;DR

PLUTO provides a structured, questionnaire-based framework grounded in data solidarity to assess the public value of specific data uses, addressing the gap between qualitative notions of public value and actionable governance. The approach combines a weighted questionnaire with a real-time visualization, implemented as a self-hosted web tool, and evaluated through iterative expert feedback, a workshop, and a formal usability study with non-experts. The results indicate strong usability and perceived explainability, while highlighting challenges in terminology, bias, and balancing comprehensive assessment with accessibility. By operationalizing public value into a pragmatic, adaptable tool, PLUTO aims to foster inclusive, accountable data governance that engages regulators, industry, NGOs, and the public. The work advances practical methodology for translating data solidarity principles into concrete, user-friendly assessment practices that can be tailored to domain contexts.

Abstract

We present PLUTO (Public VaLUe Assessment TOol), a framework for assessing the public value of specific instances of data use. Grounded in the concept of data solidarity, PLUTO aims to empower diverse stakeholders-including regulatory bodies, private enterprises, NGOs, and individuals-to critically engage with data projects through a structured assessment of the risks and benefits of data use, and by encouraging critical reflection. This paper discusses the theoretical foundation, development process, and initial user experiences with PLUTO. Key challenges include translating qualitative assessments of benefits and risks into actionable quantitative metrics while maintaining inclusivity and transparency. Initial feedback highlights PLUTO's potential to foster responsible decision-making and shared accountability in data practices.

Figures (33)

• Figure 1: The four types of data use within data solidarity framework, plotted on a risk-benefit matrix. Each quadrant defines a data use type (A-D) and prescribes a corresponding governance action—from support (Type A) to prohibition (Type D)—based on its position within the framework.
• Figure 2: Key elements for assessing the public value of data use. These elements align with the categories of questionnaire items.
• Figure 3: An overview of low-fidelity prototypes for visualizing submission results. The design highlighted in the orange frame was selected for implementation. All designs are available at https://github.com/PLUTO-UniWien/PLUTO/blob/f300cb595e3e5618c9af8f6cdcef9c83a5730144/docs/designs/Prototypes.pdf.
• Figure 5: PLUTO results interface, integrating multiple components to support user comprehension. The primary visualization is a risk-benefit matrix that plots the overall assessment, contextualized by a qualitative A-D scale (from favorable to discouraged). To enhance accessibility for users less familiar with coordinate systems, separate sliders offer a direct, one-dimensional view of the Risk and Benefit scores. For full transparency, the interface details how user answers contribute to the final result. Finally, actionable recommendations provide concrete steps for improving the project's public value by increasing benefits and mitigating risks.
• Figure 6: Maximum Reachable Points (Single Value 1): This visualization prominently displays 80% of the maximum achievable points using a straightforward and bold design. It offers a clear and concise depiction of progress, which is advantageous for quickly assessing performance. However, it does not provide additional context or metrics. By presenting data in an easily understandable format, it aligns with project goals focused on inclusivity and accessibility. While it simplifies numerical complexity and uses an intuitive metric familiar to most audiences, it does not offer the detailed insights required for more in-depth analysis.