RAISE: A Unified Framework for Responsible AI Scoring and Evaluation
Loc Phuc Truong Nguyen, Hung Thanh Do
TL;DR
RAISE introduces a unified, multi-dimensional evaluation framework for responsible AI on tabular data, bridging regulatory principles and verifiable assessment by normalizing performance with a common $F1$-Score. It aggregates 21 metrics across explainability, fairness, sustainability, and robustness into a Dimension Score per pillar and an overall Responsibility Score, while reporting predictive accuracy separately. Across three high-stakes datasets and three architectures, the study reveals consistent trade-offs: the FT-Transformer excels in fairness and attribution quality at a high energy cost, the MLP is robust and energy-efficient, and TabResNet offers a balanced profile. The framework enables context-aware model selection and governance, and its open-source implementation supports reproducible auditing and regulatory alignment.
Abstract
As AI systems enter high-stakes domains, evaluation must extend beyond predictive accuracy to include explainability, fairness, robustness, and sustainability. We introduce RAISE (Responsible AI Scoring and Evaluation), a unified framework that quantifies model performance across these four dimensions and aggregates them into a single, holistic Responsibility Score. We evaluated three deep learning models: a Multilayer Perceptron (MLP), a Tabular ResNet, and a Feature Tokenizer Transformer, on structured datasets from finance, healthcare, and socioeconomics. Our findings reveal critical trade-offs: the MLP demonstrated strong sustainability and robustness, the Transformer excelled in explainability and fairness at a very high environmental cost, and the Tabular ResNet offered a balanced profile. These results underscore that no single model dominates across all responsibility criteria, highlighting the necessity of multi-dimensional evaluation for responsible model selection. Our implementation is available at: https://github.com/raise-framework/raise.