Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computational Dualism and Objective Superintelligence

Michael Timothy Bennett

TL;DR

This work proposes a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states, and formalises systems as behaviour and cognition as embodied, embedded, extended and enactive, which suggests AGI will be safer, but more limited, than theorised.

Abstract

The concept of intelligent software is flawed. The behaviour of software is determined by the hardware that "interprets" it. This undermines claims regarding the behaviour of theorised, software superintelligence. Here we characterise this problem as "computational dualism", where instead of mental and physical substance, we have software and hardware. We argue that to make objective claims regarding performance we must avoid computational dualism. We propose a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states. We formalise systems as behaviour (inputs and outputs), and cognition as embodied, embedded, extended and enactive. The result is cognition formalised as a part of the environment, rather than as a disembodied policy interacting with the environment through an interpreter. This allows us to make objective claims regarding intelligence, which we argue is the ability to "generalise", identify causes and adapt. We then establish objective upper bounds for intelligent behaviour. This suggests AGI will be safer, but more limited, than theorised.

Computational Dualism and Objective Superintelligence

TL;DR

This work proposes a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states, and formalises systems as behaviour and cognition as embodied, embedded, extended and enactive, which suggests AGI will be safer, but more limited, than theorised.

Abstract

The concept of intelligent software is flawed. The behaviour of software is determined by the hardware that "interprets" it. This undermines claims regarding the behaviour of theorised, software superintelligence. Here we characterise this problem as "computational dualism", where instead of mental and physical substance, we have software and hardware. We argue that to make objective claims regarding performance we must avoid computational dualism. We propose a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states. We formalise systems as behaviour (inputs and outputs), and cognition as embodied, embedded, extended and enactive. The result is cognition formalised as a part of the environment, rather than as a disembodied policy interacting with the environment through an interpreter. This allows us to make objective claims regarding intelligence, which we argue is the ability to "generalise", identify causes and adapt. We then establish objective upper bounds for intelligent behaviour. This suggests AGI will be safer, but more limited, than theorised.
Paper Structure (3 sections, 1 theorem, 1 equation)

This paper contains 3 sections, 1 theorem, 1 equation.

Table of Contents

  1. Introduction
  2. Pancomputational Enactivism
  3. Limits of Intelligence

Key Result

proposition thmcounterproposition

The most 'intelligent' choice of policy and vocabulary given uninstantiated task $\lambda_\rho$ is $\pi$ and $\mathfrak{v}$ s.t. $\mathfrak{v}$ maximises utility for $\lambda_\rho(\mathfrak{v})$, $\pi \in \Pi_{\lambda_\rho(\mathfrak{v})}$ and $\pi$ maximises weakness.

Theorems & Definitions (9)

  • definition thmcounterdefinition: environment
  • definition thmcounterdefinition: abstraction layer
  • definition thmcounterdefinition: $\mathfrak{v}$-task
  • definition thmcounterdefinition: inference
  • definition thmcounterdefinition: learning
  • definition thmcounterdefinition: utility of intelligence
  • definition thmcounterdefinition: uninstantiated-tasks
  • proposition thmcounterproposition: upper bound
  • proof