Human-aligned AI Model Cards with Weighted Hierarchy Architecture

TL;DR

The paper tackles fragmentation in AI documentation amid rapid LLM growth by introducing CRAI-MCF, a Value Sensitive Design–driven framework that converts static disclosures into actionable, quantitatively evaluable documentation. It builds an empirical, practice-grounded taxonomy of 217 atomic parameters across eight modules and uses priors $s_i = \frac{f_i}{N}$ with $N=240$ and a BaselineScore$$(M) = \big(\frac{O_M}{O_{All}} + \frac{A_M}{A_{All}}\big) \cdot \frac{S_M}{2},$$ where $S_M = \sum_{i \in M} s_i$, to enable cross-model comparison and determine sufficiency. The framework demonstrates readability and like-for-like comparability improvements, while reducing documentation prose by about $38\%$ through information-parity mapping, without sacrificing verifiability. Practically, CRAI-MCF guides engineers to prioritize high-priority, evidence-backed parameters, improves governance coverage, and supports more responsible adoption of diverse AI systems across domains.

Abstract

The proliferation of Large Language Models (LLMs) has led to a burgeoning ecosystem of specialized, domain-specific models. While this rapid growth accelerates innovation, it has simultaneously created significant challenges in model discovery and adoption. Users struggle to navigate this landscape due to inconsistent, incomplete, and imbalanced documentation across platforms. Existing documentation frameworks, such as Model Cards and FactSheets, attempt to standardize reporting but are often static, predominantly qualitative, and lack the quantitative mechanisms needed for rigorous cross-model comparison. This gap exacerbates model underutilization and hinders responsible adoption. To address these shortcomings, we introduce the Comprehensive Responsible AI Model Card Framework (CRAI-MCF), a novel approach that transitions from static disclosures to actionable, human-aligned documentation. Grounded in Value Sensitive Design (VSD), CRAI-MCF is built upon an empirical analysis of 240 open-source projects, distilling 217 parameters into an eight-module, value-aligned architecture. Our framework introduces a quantitative sufficiency criterion to operationalize evaluation and enables rigorous cross-model comparison under a unified scheme. By balancing technical, ethical, and operational dimensions, CRAI-MCF empowers practitioners to efficiently assess, select, and adopt LLMs with greater confidence and operational integrity.

Figures (5)

• Figure 1: The VSD-anchored research pipeline
• Figure 2: Tree visualization of CRAI-MCF. The diagram highlights a subset of core parameters under the eight Level-0 modules, illustrating their hierarchical organization and containment relationships across Levels 1–5. The figure is schematic rather than exhaustive, focusing on structural layering rather than the full set of 217 leaf parameters.
• Figure 3: Conceptual integration of the five evaluation dimensions. Technical soundness, readability, completeness, applicability, and dynamic maintenance converge on responsible documentation.
• Figure 4: Task-module documentation heatmap. Lighter/warmer cells denote higher coverage; darker/cooler cells denote lower coverage. Persistent dark bands on Feedback and, for many tasks, Broader Implications indicate ecosystem-level under-reporting of accountability and societal-risk information.
• Figure 5: Three-domain case comparison. Bars report the proportion preferring CRAI-MCF over the original for rapid task comprehension and preferred version to consult (Wilson 95% CIs; dashed line at 0.5 marks the chance baseline).