Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeurIPS should lead scientific consensus on AI policy

Rishi Bommasani

TL;DR

The paper argues that AI policy design requires both rigorous evidence and scientific consensus, and that NeurIPS is uniquely positioned to catalyze a formal consensus process. It distinguishes consensus formation from evidence generation and synthesis, outlining NeurIPS’s internal convening power and external credibility as core strengths, while critiquing alternatives such as the IPCC-like synthesis report and politics-driven AI summits. Drawing on IPCC lessons—legitimacy, credibility, and impact—the authors propose pilots (a standing working group, a dedicated policy track, debates, and surveys) to implement and circulate consensus, plus concrete topics like evaluation selection and threshold design as initial focal points, including references to $10^{26}$ compute thresholds. They also acknowledge counterarguments and argue that NeurIPS can deliver credible, policy-relevant consensus that improves AI policy outcomes without compromising scientific integrity.

Abstract

Designing wise AI policy is a grand challenge for society. To design such policy, policymakers should place a premium on rigorous evidence and scientific consensus. While several mechanisms exist for evidence generation, and nascent mechanisms tackle evidence synthesis, we identify a complete void on consensus formation. In this position paper, we argue NeurIPS should actively catalyze scientific consensus on AI policy. Beyond identifying the current deficit in consensus formation mechanisms, we argue that NeurIPS is the best option due its strengths and the paucity of compelling alternatives. To make progress, we recommend initial pilots for NeurIPS by distilling lessons from the IPCC's leadership to build scientific consensus on climate policy. We dispel predictable counters that AI researchers disagree too much to achieve consensus and that policy engagement is not the business of NeurIPS. NeurIPS leads AI on many fronts, and it should champion scientific consensus to create higher quality AI policy.

NeurIPS should lead scientific consensus on AI policy

TL;DR

The paper argues that AI policy design requires both rigorous evidence and scientific consensus, and that NeurIPS is uniquely positioned to catalyze a formal consensus process. It distinguishes consensus formation from evidence generation and synthesis, outlining NeurIPS’s internal convening power and external credibility as core strengths, while critiquing alternatives such as the IPCC-like synthesis report and politics-driven AI summits. Drawing on IPCC lessons—legitimacy, credibility, and impact—the authors propose pilots (a standing working group, a dedicated policy track, debates, and surveys) to implement and circulate consensus, plus concrete topics like evaluation selection and threshold design as initial focal points, including references to compute thresholds. They also acknowledge counterarguments and argue that NeurIPS can deliver credible, policy-relevant consensus that improves AI policy outcomes without compromising scientific integrity.

Abstract

Designing wise AI policy is a grand challenge for society. To design such policy, policymakers should place a premium on rigorous evidence and scientific consensus. While several mechanisms exist for evidence generation, and nascent mechanisms tackle evidence synthesis, we identify a complete void on consensus formation. In this position paper, we argue NeurIPS should actively catalyze scientific consensus on AI policy. Beyond identifying the current deficit in consensus formation mechanisms, we argue that NeurIPS is the best option due its strengths and the paucity of compelling alternatives. To make progress, we recommend initial pilots for NeurIPS by distilling lessons from the IPCC's leadership to build scientific consensus on climate policy. We dispel predictable counters that AI researchers disagree too much to achieve consensus and that policy engagement is not the business of NeurIPS. NeurIPS leads AI on many fronts, and it should champion scientific consensus to create higher quality AI policy.

Paper Structure

This paper contains 25 sections, 1 figure.

Table of Contents

  1. Introduction
  2. The Problem: Consensus Formation
  3. Strengths of NeurIPS
  4. Internal role.
  5. External role.
  6. Weaknesses of alternatives
  7. International Scientific Report on AI.
  8. AI summit series.
  9. Other options.
  10. Guidance on the Process
  11. Lessons from the IPCC
  12. Legitimacy.
  13. Credibility.
  14. Impact.
  15. Pilots to implement
  16. ...and 10 more sections

Figures (1)

  • Figure 1: The IPCC Assessment Report process. To develop scientific consensus on climate policy, the IPCC conducts an elaborate multi-step process to ensure the legitimacy and credibility of the resulting scientific consensus.