Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The hardware is the software

Jeremie Laydevant, Logan G. Wright, Tianyu Wang, Peter L. McMahon

TL;DR

AI hardware is constrained by substrate physics, not abstract software. The authors argue for physics-agnostic principles of biological intelligence that can be ported across substrates, and critique attempts to mimic biology without accounting for hardware. They review how hardware has historically shaped AI algorithms (e.g., GPUs, Deep Blue) and advocate translating biological strategies through the physics of the target substrate. They propose a roadmap for neuroscience-hardware co-design, emphasizing universal principles and cross-disciplinary collaboration to advance neuromorphic AI.

Abstract

Human brains and bodies are not hardware running software: the hardware is the software. We reason that because the microscopic physics of artificial-intelligence hardware and of human biological "hardware" is distinct, neuromorphic engineers need to be cautious (and yet also creative) in how we take inspiration from biological intelligence. We should focus primarily on principles and design ideas that respect -- and embrace -- the underlying hardware physics of non-biological intelligent systems, rather than abstracting it away. We see a major role for neuroscience in neuromorphic computing as identifying the physics-agnostic principles of biological intelligence -- that is the principles of biological intelligence that can be gainfully adapted and applied to any physical hardware.

The hardware is the software

TL;DR

AI hardware is constrained by substrate physics, not abstract software. The authors argue for physics-agnostic principles of biological intelligence that can be ported across substrates, and critique attempts to mimic biology without accounting for hardware. They review how hardware has historically shaped AI algorithms (e.g., GPUs, Deep Blue) and advocate translating biological strategies through the physics of the target substrate. They propose a roadmap for neuroscience-hardware co-design, emphasizing universal principles and cross-disciplinary collaboration to advance neuromorphic AI.

Abstract

Human brains and bodies are not hardware running software: the hardware is the software. We reason that because the microscopic physics of artificial-intelligence hardware and of human biological "hardware" is distinct, neuromorphic engineers need to be cautious (and yet also creative) in how we take inspiration from biological intelligence. We should focus primarily on principles and design ideas that respect -- and embrace -- the underlying hardware physics of non-biological intelligent systems, rather than abstracting it away. We see a major role for neuroscience in neuromorphic computing as identifying the physics-agnostic principles of biological intelligence -- that is the principles of biological intelligence that can be gainfully adapted and applied to any physical hardware.
Paper Structure (6 sections, 1 figure)

This paper contains 6 sections, 1 figure.

Table of Contents

  1. Prologue: Life, with lasers
  2. The Hardware is the Software: Introduction
  3. A core theme of neuromorphic computing has been to force non-biological systems to act like biological systems
  4. Physical constraints define both our brains and modern AI systems
  5. Neural principles have nonetheless shaped modern AI, but only where they are compatible with the underlying constraints of physical hardware.
  6. Conclusions and looking forward

Figures (1)

  • Figure 1: The available hardware determines what and how intelligence can be engineered.a. Deep-sea creatures on a far-away planet think and communicate using light. b. Different physics leads to different attributes of computing hardware, such as the signal-to-noise ratio (SNR) of information-carrying signals; update rate---the speed of the hardware's dynamics, which affects how quickly calculations can be performed; and dimensionality---how many spatial dimensions the hardware can take form in. Different algorithms work formidably well in the brain and on digital electronics because they leverage the hardware at its best. We expect that hardware-agnostic principles of intelligence would also map to unconventional hardware such as optics or analog electronics. c. The hardware-shaped evolution of AI algorithms. Performance breakthroughs on specific tasks have tended to rely on software that efficiently exploits the strengths of the available computing machinery. This observation---that the best algorithms are those that best exploit the available hardware---has been made several times before rodney1999cambriansutton2019bitterhooker2020lottery, and described as "the bitter lesson" sutton2019bitter and "the hardware lottery" hooker2020lottery.