Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts

TL;DR

The paper addresses why AI experts disagree on existential risk and P(doom) by surveying 111 AI professionals on familiarity with AI safety concepts, exposure to safety arguments, and reactions to interventions. It identifies two persistent worldviews—AI as a controllable tool and AI as an uncontrollable agent—that correlate with risk perception and AGI development timelines, and it documents substantial gaps in familiarity with core AI safety concepts. Despite high overall concern about catastrophic risk (77%), many experts lack knowledge of foundational safety ideas (e.g., instrumental convergence is unfamiliar to about 63%), underscoring a safety-literacy gap that likely drives skepticism. Interventions yield only modest opinion shifts, suggesting stable worldviews and highlighting the need for foundational safety education to improve cross-domain dialogue and policy-relevant forecasting.

Abstract

The development of artificial general intelligence (AGI) is likely to be one of humanity's most consequential technological advancements. Leading AI labs and scientists have called for the global prioritization of AI safety citing existential risks comparable to nuclear war. However, research on catastrophic risks and AI alignment is often met with skepticism, even by experts. Furthermore, online debate over the existential risk of AI has begun to turn tribal (e.g. name-calling such as "doomer" or "accelerationist"). Until now, no systematic study has explored the patterns of belief and the levels of familiarity with AI safety concepts among experts. I surveyed 111 AI experts on their familiarity with AI safety concepts, key objections to AI safety, and reactions to safety arguments. My findings reveal that AI experts cluster into two viewpoints -- an "AI as controllable tool" and an "AI as uncontrollable agent" perspective -- diverging in beliefs toward the importance of AI safety. While most experts (78%) agreed or strongly agreed that "technical AI researchers should be concerned about catastrophic risks", many were unfamiliar with specific AI safety concepts. For example, only 21% of surveyed experts had heard of "instrumental convergence," a fundamental concept in AI safety predicting that advanced AI systems will tend to pursue common sub-goals (such as self-preservation). The least concerned participants were the least familiar with concepts like this, suggesting that effective communication of AI safety should begin with establishing clear conceptual foundations in the field.

Figures (11)

• Figure 1: Pie chart of respondent career: N=111. Participants were asked "What best describes what you are currently working on?" Options included: "Academic researcher in AI/ML/related field," "Industry engineer or researcher in AI/ML/related field," and "AI safety researcher or professional."
• Figure 2: Bar Chart of Desired AGI Timelines Participants were asked "Which best describes your position on when we should build AGI?" The participants had the following options: "We should never build AGI," "Eventually, but not soon," "Soon, but not as fast as possible," "We should develop more powerful and more general systems as fast as possible." Participants were split by their career.
• Figure 3: Network of Correlations Between Each Belief: Participants were asked to rate their agreement with various AGI-related statements on a 1-5 Likert scale (1=strongly disagree, 5=strongly agree). These statements were chosen because they were found to have the strongest correlations. Edges between nodes are only placed for correlations R=>.25 and p < .05. Boldness corresponds to the magnitude the correlation. Higher values for the timelines value correspond to shorter desired timelines (e.g. ASAP or "soon"). More detailed data can be found in Appendix \ref{['appendix:correlations_sec']}.
• Figure 4: Radar Chart of Familiarity with Concepts Across Groups: The figure shows familiarity ratings (0-4 scale) for ML concepts (on top in red) and AI safety concepts (in green, on bottom) I translated qualitative data (5 point Likert scale) of familiarity into quantitative data with the following mapping: 'Never heard of it': 0, 'Heard of it': 1, 'Know a little': 2, 'Know a fair amount': 3, 'Know it well': 4. AI safety experts show higher familiarity with safety concepts but slightly lower familiarity with empirical ML concepts. N=111 total respondents (Academic=66%, Industry=16%, AI Safety=9%).
• Figure 5: Correlations Between AI Safety Knowledge and "AI-as-a-tool" beliefs: I map Pearson correlation coefficients between participants' familiarity with key AI safety concepts and their beliefs about AI. The full statements can be found in Table \ref{['tab:ai_safety_statements']}. * p < 0.05, ** p < 0.01, *** p < 0.001.