Securing External Deeper-than-black-box GPAI Evaluations

TL;DR

The paper argues that external GPAI evaluations must move beyond traditional black-box testing by enabling structured access to model internals and data lineage to uncover latent risks and enhance accountability. It offers a taxonomy of evaluation modalities from black-box to white-box and details techniques across mechanistic interpretability, robustness, gradient analysis, privacy, and reasoning verification, alongside security considerations for remote assessments. A combined technical and legal safeguards framework is proposed, including secure enclaves, differential privacy, zero-knowledge proofs, blockchain provenance, and RBAC, as well as on-site evaluations as an ideal but currently challenging approach. Policy implications are discussed, advocating a hybrid governance model, national safety institutes, international coordination, and standardization to enable scalable, trustworthy external audits. The work aims to improve public trust and safety while advancing AI interpretability and governance research.

Abstract

This paper examines the critical challenges and potential solutions for conducting secure and effective external evaluations of general-purpose AI (GPAI) models. With the exponential growth in size, capability, reach and accompanying risk of these models, ensuring accountability, safety, and public trust requires frameworks that go beyond traditional black-box methods. The discussion begins with an analysis of the need for deeper-than-black-box evaluations (Section I), emphasizing the importance of understanding model internals to uncover latent risks and ensure compliance with ethical and regulatory standards. Building on this foundation, Section II addresses the security considerations of remote evaluations, outlining the threat landscape, technical solutions, and safeguards necessary to protect both evaluators and proprietary model data. Finally, Section III synthesizes these insights into actionable recommendations and future directions, aiming to establish a robust, scalable, and transparent framework for external assessments in GPAI governance.

Figures (3)

• Figure 1: Different access-specific AI evaluation techniques. From completely black-box access (left-most part of the diagram) to completely white-box access (right-most part), there is a wide variety of SOTA techniques that an external evaluator could perform depending on the level of model access that they are granted. We colour-code techniques belonging to the same group (see in-figure legend), even if there is a clear ordering of model access that is needed to perform each of these (e.g. Robustness Testing (red) requires almost completely black-box access, while Circuit-Level Interpretability (dark blue) needs white-box access).
• Figure 2: example scenarios from the threat landscape when performing remote AI model evaluations. We map different relevant threats onto two axes: Organizational Capacity (following work from nevo2024securing) and Intentionality (to which degree the relevant attack vector needs a motivated actor to take place).
• Figure 3: a graphical summary of how the Structured Transparency framework trask2020beyond can look in practice. There are five key elements to guaranteeing that the flow of information goes only from the sender to the target receiver. For each aspect of the framework, we list all of the mitigations we explain in the main text.