Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Standardized Machine-readable Dataset Documentation Format for Responsible AI

Nitisha Jain, Mubashara Akhtar, Joan Giner-Miguelez, Rajat Shinde, Joaquin Vanschoren, Steffen Vogler, Sujata Goswami, Yuhan Rao, Tim Santos, Luis Oala, Michalis Karamousadakis, Manil Maskey, Pierre Marcenac, Costanza Conforti, Michael Kuchnik, Lora Aroyo, Omar Benjelloun, Elena Simperl

TL;DR

Data documentation quality is a bottleneck for responsible AI. This paper proposes Croissant-RAI, a machine-readable extension of Croissant based on Schema.org that standardizes RAI metadata for AI datasets. The design is driven by five use cases (life cycle, labeling, participatory data, safety/fairness evaluation, regulatory compliance) and demonstrated via real datasets, with a Python library and a visual editor to facilitate adoption. The approach aims to improve discoverability, interoperability, and trustworthy data reuse by data publishers, search engines, and ML workflows, enabling more robust and compliant AI systems.

Abstract

Data is critical to advancing AI technologies, yet its quality and documentation remain significant challenges, leading to adverse downstream effects (e.g., potential biases) in AI applications. This paper addresses these issues by introducing Croissant-RAI, a machine-readable metadata format designed to enhance the discoverability, interoperability, and trustworthiness of AI datasets. Croissant-RAI extends the Croissant metadata format and builds upon existing responsible AI (RAI) documentation frameworks, offering a standardized set of attributes and practices to facilitate community-wide adoption. Leveraging established web-publishing practices, such as Schema.org, Croissant-RAI enables dataset users to easily find and utilize RAI metadata regardless of the platform on which the datasets are published. Furthermore, it is seamlessly integrated into major data search engines, repositories, and machine learning frameworks, streamlining the reading and writing of responsible AI metadata within practitioners' existing workflows. Croissant-RAI was developed through a community-led effort. It has been designed to be adaptable to evolving documentation requirements and is supported by a Python library and a visual editor.

A Standardized Machine-readable Dataset Documentation Format for Responsible AI

TL;DR

Data documentation quality is a bottleneck for responsible AI. This paper proposes Croissant-RAI, a machine-readable extension of Croissant based on Schema.org that standardizes RAI metadata for AI datasets. The design is driven by five use cases (life cycle, labeling, participatory data, safety/fairness evaluation, regulatory compliance) and demonstrated via real datasets, with a Python library and a visual editor to facilitate adoption. The approach aims to improve discoverability, interoperability, and trustworthy data reuse by data publishers, search engines, and ML workflows, enabling more robust and compliant AI systems.

Abstract

Data is critical to advancing AI technologies, yet its quality and documentation remain significant challenges, leading to adverse downstream effects (e.g., potential biases) in AI applications. This paper addresses these issues by introducing Croissant-RAI, a machine-readable metadata format designed to enhance the discoverability, interoperability, and trustworthiness of AI datasets. Croissant-RAI extends the Croissant metadata format and builds upon existing responsible AI (RAI) documentation frameworks, offering a standardized set of attributes and practices to facilitate community-wide adoption. Leveraging established web-publishing practices, such as Schema.org, Croissant-RAI enables dataset users to easily find and utilize RAI metadata regardless of the platform on which the datasets are published. Furthermore, it is seamlessly integrated into major data search engines, repositories, and machine learning frameworks, streamlining the reading and writing of responsible AI metadata within practitioners' existing workflows. Croissant-RAI was developed through a community-led effort. It has been designed to be adaptable to evolving documentation requirements and is supported by a Python library and a visual editor.
Paper Structure (20 sections, 3 figures, 2 tables)

This paper contains 20 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Machine-readable metadata
  4. Croissant: A metadata format for ML-ready datasets
  5. Responsible AI data documentation
  6. Croissant-RAI Vocabulary Engineering
  7. Use Cases
  8. Use case 1: The data life cycle.
  9. Use case 2: Data labeling.
  10. Use case 3: Participatory data.
  11. Use case 4: AI safety and fairness evaluation.
  12. Use case 5: Regulatory compliance
  13. The Responsible AI Vocabulary
  14. Application examples
  15. RAI Geospatial AI-ready datasets
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The Multi-step Vocabulary Engineering Process for Croissant-RAI
  • Figure 3: Overview of the Croissant-RAI vocabulary
  • Figure 4: UI of the RAI extension of the Croissant editor