Specification Overfitting in Artificial Intelligence

TL;DR

This paper defines specification overfitting as the tendency of AI systems to optimize additional specification metrics at the expense of core task performance or other requirements. It conducts a large-scale literature survey (2018–mid-2023) across NLP, CV, RL, and related areas, identifying 74 papers that propose or optimize specification metrics and revealing a lack of guidance on how specifications should influence system development or be scoped. The authors categorize specification types (robustness, fairness, capabilities), specification encodings (example-based vs metric-based), and optimization strategies (direct/indirect/no optimization), and they analyze evaluation schemes for overfitting, reporting that most work neglects explicit development-process roles and scope. They illustrate the regulatory relevance by mapping to the EU AI Act and harmonized standards, underscoring the risk that spec-focused optimization could undermine real-world performance or fairness if not properly managed. The work calls for explicit, robust guidelines and evaluation schemes that consider multi-metric trade-offs, to ensure regulatory compliance without sacrificing overall system quality and safety.

Abstract

Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this technology's potential negative side effects. High-level requirements such as fairness and robustness need to be formalized into concrete specification metrics, imperfect proxies that capture isolated aspects of the underlying requirements. Given possible trade-offs between different metrics and their vulnerability to over-optimization, integrating specification metrics in system development processes is not trivial. This paper defines specification overfitting, a scenario where systems focus excessively on specified metrics to the detriment of high-level requirements and task performance. We present an extensive literature survey to categorize how researchers propose, measure, and optimize specification metrics in several AI fields (e.g., natural language processing, computer vision, reinforcement learning). Using a keyword-based search on papers from major AI conferences and journals between 2018 and mid-2023, we identify and analyze 74 papers that propose or optimize specification metrics. We find that although most papers implicitly address specification overfitting (e.g., by reporting more than one specification metric), they rarely discuss which role specification metrics should play in system development or explicitly define the scope and assumptions behind metric formulations.

Figures (1)

• Figure 1: Number of relevant papers by year (left) and application area (right).