An AI System Evaluation Framework for Advancing AI Safety: Terminology, Taxonomy, Lifecycle Mapping
Boming Xia, Qinghua Lu, Liming Zhu, Zhenchang Xing
TL;DR
The paper addresses the mismatch between safety evaluations across AI, software engineering, and governance, arguing that model-centric evaluation misses the broader AI system and its environmental context. It proposes a framework with three components: harmonised terminology, a comprehensive taxonomy for component- and system-level evaluation, and lifecycle-based mapping to stakeholders, to enable holistic assessment of AI safety. The taxonomy covers non-AI components, data, models, and guardrails at the component level, and expands to system-level evaluation for Narrow and General AI, including guardrails and environmental affordances. By mapping evaluations to lifecycle stages and multiple stakeholders, the framework aims to improve transparency, accountability, and safety across the AI supply chain and deployments.
Abstract
The advent of advanced AI underscores the urgent need for comprehensive safety evaluations, necessitating collaboration across communities (i.e., AI, software engineering, and governance). However, divergent practices and terminologies across these communities, combined with the complexity of AI systems-of which models are only a part-and environmental affordances (e.g., access to tools), obstruct effective communication and comprehensive evaluation. This paper proposes a framework for AI system evaluation comprising three components: 1) harmonised terminology to facilitate communication across communities involved in AI safety evaluation; 2) a taxonomy identifying essential elements for AI system evaluation; 3) a mapping between AI lifecycle, stakeholders, and requisite evaluations for accountable AI supply chain. This framework catalyses a deeper discourse on AI system evaluation beyond model-centric approaches.