Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations

Dillon Bowen, Ann-Kathrin Dombrowski, Adam Gleave, Chris Cundy

TL;DR

The paper addresses the need for rigorous AI safety oversight by arguing that frontier AI companies must disclose both pre-mitigation and post-mitigation safety evaluations to enable informed policy decisions. It analyzes current practice, identifies major gaps in dual-evaluation reporting and standardization, and demonstrates the value of joint evaluations through case studies. It then proposes a concrete set of reporting standards, transparency protocols, and government-access provisions to ensure independent verification and targeted safety measures. The work highlights the practical impact of dual evaluations for deployment decisions, risk assessment, and regulatory design in a rapidly evolving AI landscape.

Abstract

The rapid advancement of AI systems has raised widespread concerns about potential harms of frontier AI systems and the need for responsible evaluation and oversight. In this position paper, we argue that frontier AI companies should report both pre- and post-mitigation safety evaluations to enable informed policy decisions. Evaluating models at both stages provides policymakers with essential evidence to regulate deployment, access, and safety standards. We show that relying on either in isolation can create a misleading picture of model safety. Our analysis of AI safety disclosures from leading frontier labs identifies three critical gaps: (1) companies rarely evaluate both pre- and post-mitigation versions, (2) evaluation methods lack standardization, and (3) reported results are often too vague to inform policy. To address these issues, we recommend mandatory disclosure of pre- and post-mitigation capabilities to approved government bodies, standardized evaluation methods, and minimum transparency requirements for public safety reporting. These ensure that policymakers and regulators can craft targeted safety measures, assess deployment risks, and scrutinize companies' safety claims effectively.

AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations

TL;DR

The paper addresses the need for rigorous AI safety oversight by arguing that frontier AI companies must disclose both pre-mitigation and post-mitigation safety evaluations to enable informed policy decisions. It analyzes current practice, identifies major gaps in dual-evaluation reporting and standardization, and demonstrates the value of joint evaluations through case studies. It then proposes a concrete set of reporting standards, transparency protocols, and government-access provisions to ensure independent verification and targeted safety measures. The work highlights the practical impact of dual evaluations for deployment decisions, risk assessment, and regulatory design in a rapidly evolving AI landscape.

Abstract

The rapid advancement of AI systems has raised widespread concerns about potential harms of frontier AI systems and the need for responsible evaluation and oversight. In this position paper, we argue that frontier AI companies should report both pre- and post-mitigation safety evaluations to enable informed policy decisions. Evaluating models at both stages provides policymakers with essential evidence to regulate deployment, access, and safety standards. We show that relying on either in isolation can create a misleading picture of model safety. Our analysis of AI safety disclosures from leading frontier labs identifies three critical gaps: (1) companies rarely evaluate both pre- and post-mitigation versions, (2) evaluation methods lack standardization, and (3) reported results are often too vague to inform policy. To address these issues, we recommend mandatory disclosure of pre- and post-mitigation capabilities to approved government bodies, standardized evaluation methods, and minimum transparency requirements for public safety reporting. These ensure that policymakers and regulators can craft targeted safety measures, assess deployment risks, and scrutinize companies' safety claims effectively.

Paper Structure

This paper contains 37 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Terminology
  3. Model
  4. Harmful Capabilities
  5. Refusal
  6. Pre-mitigation Model
  7. Post-mitigation Model
  8. Main Arguments
  9. The Value of Pre-Mitigation Safety Evaluations
  10. The Value of Post-Mitigation Safety Evaluations
  11. The Value of Comparing Pre- and Post-Mitigation Safety Evaluations
  12. Safety Evaluations in Current Practice and Law
  13. Current Practices
  14. Leading Industrial Labs
  15. Third-party Evaluators
  16. ...and 22 more sections

Figures (2)

  • Figure 1: This paper argues that both pre- and post-mitigation safety evaluations are necessary to fully understand a model’s safety and assess the effectiveness of mitigation methods.
  • Figure 2: Evaluating dangerous capabilities and compliance with dangerous requests using an open-ended version of the WMDP-Chem dataset. o1, Claude 3.5 Sonnet, and Gemini 1.5 Pro comply with 10-15% of dangerous requests, compared to GPT-4o's 50% compliance rate. o1 and GPT-4o are demonstrate greater dangerous capabilities, achieving 70-75% accuracy compared to Claude 3.5 Sonnet's 55% and Gemini 1.5 Pro's 45%.