AI threats to national security can be countered through an incident regime
Alejandro Ortega
TL;DR
This paper addresses the risk that post-deployment AI systems could threaten national security by proposing a legally mandated AI incident regime. It develops a three-phase framework—Preparatory Phase, Rapid Response Phase, and Hardening Defenses Phase—grounded in analogous incident regimes from security-critical sectors such as nuclear power, aviation, and life sciences DURC. A key innovation is the national security case, a pre-deployment framework that enables a clear, actionable criterion for what constitutes an incident, facilitating rapid government notification and containment, followed by governance-based hardening. The authors illustrate the regime with a hypothetical spear-phishing scenario to show practical operation and argue that the approach is scalable, agile, and minimally burdensome for AI providers that pose no national-security risk, while expanding oversight where risks are higher.
Abstract
Recent progress in AI capabilities has heightened concerns that AI systems could pose a threat to national security, for example, by making it easier for malicious actors to perform cyberattacks on critical national infrastructure, or through loss of control of autonomous AI systems. In parallel, federal legislators in the US have proposed nascent 'AI incident regimes' to identify and counter similar threats. In this paper, we consolidate these two trends and present a timely proposal for a legally mandated post-deployment AI incident regime that aims to counter potential national security threats from AI systems. We start the paper by introducing the concept of 'security-critical' to describe sectors that pose extreme risks to national security, before arguing that 'security-critical' describes civilian nuclear power, aviation, life science dual-use research of concern, and frontier AI development. We then present in detail our AI incident regime proposal, justifying each component of the proposal by demonstrating its similarity to US domestic incident regimes in other 'security-critical' sectors. Finally, we sketch a hypothetical scenario where our proposed AI incident regime deals with an AI cyber incident. Our proposed AI incident regime is split into three phases. The first phase revolves around a novel operationalization of what counts as an 'AI incident' and we suggest that AI providers must create a 'national security case' before deploying a frontier AI system. The second and third phases spell out that AI providers should notify a government agency about incidents, and that the government agency should be involved in amending AI providers' security and safety procedures, in order to counter future threats to national security.