AI Consciousness and Existential Risk

TL;DR

It is explained that while intelligence is a direct predictor of an AI system's existential threat, consciousness is not, and recognition of these distinctions can help AI safety researchers and public policymakers focus on the most pressing issues.

Abstract

In AI, the existential risk denotes the hypothetical threat posed by an artificial system that would possess both the capability and the objective, either directly or indirectly, to eradicate humanity. This issue is gaining prominence in scientific debate due to recent technical advancements and increased media coverage. In parallel, AI progress has sparked speculation and studies about the potential emergence of artificial consciousness. The two questions, AI consciousness and existential risk, are sometimes conflated, as if the former entailed the latter. Here, I explain that this view stems from a common confusion between consciousness and intelligence. Yet these two properties are empirically and theoretically distinct. Arguably, while intelligence is a direct predictor of an AI system's existential threat, consciousness is not. There are, however, certain incidental scenarios in which consciousness could influence existential risk, in either direction. Consciousness could be viewed as a means towards AI alignment, thereby lowering existential risk; or, it could be a precondition for reaching certain capabilities or levels of intelligence, and thus positively related to existential risk. Recognizing these distinctions can help AI safety researchers and public policymakers focus on the most pressing issues.

Figures (2)

• Figure 1: Intelligence and consciousness are two separate dimensions. Current AI models are low on both axes. The white dots are not meant to reflect actual measurements on existing AI models, but only as an illustration of this paper's message. The term "zombie" originates from the philosophy of mind literature, and denotes a hypothetical entity with no inner experience, possibly despite exterior manifestations of a normally conscious cognition. To a first approximation, existential risk (illustrated by the color scale) is a monotonically increasing function of intelligence but does not depend on consciousness.
• Figure 2: Secondhand x-risk from AI consciousness. Two specific scenarios could indirectly produce an effective correlation between consciousness level and existential risk. In (a) Alignment by consciousness, conscious AI models are exempt from x-risk because they would be intrinsically aligned with our moral and ethical values. In (b) Conscious supremacy, the same algorithms that are required for consciousness also happen to be necessary for any sufficiently advanced intelligence---so the high-intelligence/low-consciousness region of the plot would be proscribed.