Enhancements for Developing a Comprehensive AI Fairness Assessment Standard

TL;DR

This work addresses the need for robust AI fairness in high-stakes domains and emerging 6G telecom contexts, noting that existing standards focus mainly on structured data. It proposes extending the TEC Standard to cover image data, unstructured text, and generative AI (LLMs), while preserving its core three-step process and combined metrics. Key contributions include image fairness methods (tabular reduction, low-dimensional representations, cross-dataset checks, XAI), text fairness techniques (WEAT, SEAT, GBETs), and LLM-focused bias evaluation through embeddings and generation-based metrics. The enhanced framework aims to enable responsible, transparent, and trustworthy AI deployment across sectors, with ongoing updates to keep pace with evolving AI technologies.

Abstract

As AI systems increasingly influence critical sectors like telecommunications, finance, healthcare, and public services, ensuring fairness in decision-making is essential to prevent biased or unjust outcomes that disproportionately affect vulnerable entities or result in adverse impacts. This need is particularly pressing as the industry approaches the 6G era, where AI will drive complex functions like autonomous network management and hyper-personalized services. The TEC Standard for Fairness Assessment and Rating of AI Systems provides guidelines for evaluating fairness in AI, focusing primarily on tabular data and supervised learning models. However, as AI applications diversify, this standard requires enhancement to strengthen its impact and broaden its applicability. This paper proposes an expansion of the TEC Standard to include fairness assessments for images, unstructured text, and generative AI, including large language models, ensuring a more comprehensive approach that keeps pace with evolving AI technologies. By incorporating these dimensions, the enhanced framework will promote responsible and trustworthy AI deployment across various sectors.