Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incompleteness of AI Safety Verification via Kolmogorov Complexity

Munawar Hasan

Abstract

Ensuring that artificial intelligence (AI) systems satisfy formal safety and policy constraints is a central challenge in safety-critical domains. While limitations of verification are often attributed to combinatorial complexity and model expressiveness, we show that they arise from intrinsic information-theoretic limits. We formalize policy compliance as a verification problem over encoded system behaviors and analyze it using Kolmogorov complexity. We prove an incompleteness result: for any fixed sound computably enumerable verifier, there exists a threshold beyond which true policy-compliant instances cannot be certified once their complexity exceeds that threshold. Consequently, no finite formal verifier can certify all policy-compliant instances of arbitrarily high complexity. This reveals a fundamental limitation of AI safety verification independent of computational resources, and motivates proof-carrying approaches that provide instance-level correctness guarantees.

Incompleteness of AI Safety Verification via Kolmogorov Complexity

Abstract

Ensuring that artificial intelligence (AI) systems satisfy formal safety and policy constraints is a central challenge in safety-critical domains. While limitations of verification are often attributed to combinatorial complexity and model expressiveness, we show that they arise from intrinsic information-theoretic limits. We formalize policy compliance as a verification problem over encoded system behaviors and analyze it using Kolmogorov complexity. We prove an incompleteness result: for any fixed sound computably enumerable verifier, there exists a threshold beyond which true policy-compliant instances cannot be certified once their complexity exceeds that threshold. Consequently, no finite formal verifier can certify all policy-compliant instances of arbitrarily high complexity. This reveals a fundamental limitation of AI safety verification independent of computational resources, and motivates proof-carrying approaches that provide instance-level correctness guarantees.

Paper Structure

This paper contains 20 sections, 6 theorems, 15 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Preliminaries and Problem Formulation
  5. Kolmogorov Complexity
  6. Policy-Compliance Model
  7. Proof System
  8. Kolmogorov Complexity Limits of Policy Verification
  9. High-Complexity Policy-Compliant Instances
  10. Incompleteness of Policy Verification
  11. Implications for Verifiable AI
  12. Adaptation to Autonomous Systems
  13. Connection to Verifiable Perception Systems
  14. Conclusion
  15. Appendix: Incompleteness of Policy Verification
  16. ...and 5 more sections

Key Result

Lemma 1

Under Assumption ass:rich, for infinitely many $m$, there exists $x$ such that $\blacktriangleleft$$\blacktriangleleft$

Figures (1)

  • Figure 1: Verification Paradigm: A fixed verifier $T$ may fail to certify policy-compliant instances due to fundamental incompleteness, while proof-carrying approaches attach an instance-specific proof $\pi$ enabling verifiable acceptance.

Theorems & Definitions (13)

  • Definition 1: Policy-compliance
  • Definition 2: Formal verifier
  • Lemma 1: High-complexity compliant instances
  • proof
  • Theorem 1: Incompleteness of policy verification
  • proof : Proof sketch
  • Corollary 1: Limit of complete policy verification
  • Lemma 2: Counting bound
  • proof
  • Lemma 3: Existence of incompressible strings
  • ...and 3 more