Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-World Gaps in AI Governance Research

Ilan Strauss, Isobel Moure, Tim O'Reilly, Sruly Rosenblat

TL;DR

The paper investigates a growing governance gap in AI safety research by analyzing 1,178 safety & reliability papers drawn from 9,439 generative-AI papers published between 2020 and March 2025, spanning corporate labs (Anthropic, Google DeepMind, Meta, Microsoft, OpenAI) and leading universities. It shows a strong corporate emphasis on pre-deployment risks (alignment and testing) with minimal attention to post-deployment, high-risk issues (healthcare, finance, misinformation, IP, and behavioral risks). The authors attribute this to commercial incentives and significant data-access asymmetries, arguing for structured external access to deployment telemetry and an observability framework akin to LLMops to enable independent risk assessment. They propose tiered data-disclosure mechanisms, liability safe harbors, and standardized telemetry data streams to balance transparency with confidentiality, aiming to bolster public oversight and regulatory capability in deployed AI systems.

Abstract

Drawing on 1,178 safety and reliability papers from 9,439 generative AI papers (January 2020 - March 2025), we compare research outputs of leading AI companies (Anthropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of Washington). We find that corporate AI research increasingly concentrates on pre-deployment areas -- model alignment and testing & evaluation -- while attention to deployment-stage issues such as model bias has waned. Significant research gaps exist in high-risk deployment domains, including healthcare, finance, misinformation, persuasive and addictive features, hallucinations, and copyright. Without improved observability into deployed AI, growing corporate concentration could deepen knowledge deficits. We recommend expanding external researcher access to deployment data and systematic observability of in-market AI behaviors.

Real-World Gaps in AI Governance Research

TL;DR

The paper investigates a growing governance gap in AI safety research by analyzing 1,178 safety & reliability papers drawn from 9,439 generative-AI papers published between 2020 and March 2025, spanning corporate labs (Anthropic, Google DeepMind, Meta, Microsoft, OpenAI) and leading universities. It shows a strong corporate emphasis on pre-deployment risks (alignment and testing) with minimal attention to post-deployment, high-risk issues (healthcare, finance, misinformation, IP, and behavioral risks). The authors attribute this to commercial incentives and significant data-access asymmetries, arguing for structured external access to deployment telemetry and an observability framework akin to LLMops to enable independent risk assessment. They propose tiered data-disclosure mechanisms, liability safe harbors, and standardized telemetry data streams to balance transparency with confidentiality, aiming to bolster public oversight and regulatory capability in deployed AI systems.

Abstract

Drawing on 1,178 safety and reliability papers from 9,439 generative AI papers (January 2020 - March 2025), we compare research outputs of leading AI companies (Anthropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of Washington). We find that corporate AI research increasingly concentrates on pre-deployment areas -- model alignment and testing & evaluation -- while attention to deployment-stage issues such as model bias has waned. Significant research gaps exist in high-risk deployment domains, including healthcare, finance, misinformation, persuasive and addictive features, hallucinations, and copyright. Without improved observability into deployed AI, growing corporate concentration could deepen knowledge deficits. We recommend expanding external researcher access to deployment data and systematic observability of in-market AI behaviors.
Paper Structure (16 sections, 8 figures, 4 tables)

This paper contains 16 sections, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation, Data, and Methods
  3. Pre- versus post-deployment research
  4. Why commercial incentives may drive research gaps
  5. Data access challenges for independent research
  6. Data collection and sample construction
  7. Findings
  8. Corporate vs. academic generative AI research
  9. Post-deployment research gaps
  10. Policy Discussion
  11. Conclusion
  12. Appendix
  13. Additional Analysis
  14. Research Dataset Construction
  15. Classification Process: Categories
  16. ...and 1 more sections

Figures (8)

  • Figure 1: Total Citations for Safety & Reliability Research
  • Figure 2: Number of AI Safety & Reliability Papers
  • Figure 3: All Generative AI Publications by Institution (2020-2024)
  • Figure 4: AI Governance Areas by Total Paper Numbers (by Year) - Top Graph; and by Total Citations (Fractionally Adjusted) - Bottom Graph.
  • Figure : Note: Fractionally adjusted for each institution's relative authorship contribution to each paper. Not showing numbers for a category with less than 150 citations. The eight categories are chosen and defined by authors and then categorized using GPT 4o-mini. See Appendix \ref{['appendix:classification']} for definitions.
  • ...and 3 more figures