More than MACs: Exploring the Role of Neuromorphic Engineering in the Age of LLMs

TL;DR

The paper argues that to address the limitations of current large language models in real-world, embedded settings, neuromorphic and biology-inspired approaches should go beyond optimizing MACs and instead target memory integration, in-situ learning, and reduced analog-digital conversions. It provides a detailed, multi-scale comparison between biological and artificial computation across synapses, neurons, modules, and systems, highlighting where NI offers advantages and where AI currently dominates. By outlining three vectors—software, hardware, and research—it proposes concrete directions such as in-memory compute, analog and resistive memories, and agentic, grounded AI frameworks to enable embedded, adaptable AI. The work emphasizes the need for co-design, new benchmarks, and cross-disciplinary collaboration to translate NI principles into practical, scalable AI hardware and software with real-world impact.

Abstract

The introduction of large language models has significantly expanded global demand for computing; addressing this growing demand requires novel approaches that introduce new capabilities while addressing extant needs. Although inspiration from biological systems served as the foundation on which modern artificial intelligence (AI) was developed, many modern advances have been made without clear parallels to biological computing. As a result, the ability of techniques inspired by ``natural intelligence'' (NI) to inflect modern AI systems may be questioned. However, by analyzing remaining disparities between AI and NI, we argue that further biological inspiration can contribute towards expanding the capabilities of artificial systems, enabling them to succeed in real-world environments and adapt to niche applications. To elucidate which NI mechanisms can contribute toward this goal, we review and compare elements of biological and artificial computing systems, emphasizing areas of NI that have not yet been effectively captured by AI. We then suggest areas of opportunity for NI-inspired mechanisms that can inflect AI hardware and software.

Figures (11)

• Figure 1: Estimates of the average efficiency of humans at processing natural language derived from speech rates and metabolic limits suggest that LLMs running on commodity GPU hardware currently reach similar or greater efficiencies. Estimates for LLM efficiency are sourced from 12 LLMs publicly benchmarked via ML.Energy mlenergy-neuripsdb25.
• Figure 2: By leveraging alternative coding strategies for precision of weights and activations, AI accelerators have made large strides in per-op efficiencies, and some approach estimated per-synapse efficiencies for the human brain. Data from 170 AI accelerators is sourced from the Lincoln AI Computing Survey reuther_lincoln_2023.
• Figure 3: Computational systems optimized for execution of LLMs and other parallel programs contain variable amounts of memory, from megabytes to gigabytes. Available memory often corresponds with computational throughput, with exceptions including Groq's LPU and Facebook's MTIA (left). While the largest of these 170 systems can store 288 GB of data, synapses in the human brain encode 200-2000 times more information.
• Figure 4: "Batching" multiple inputs together to be calculated simultaneously in a parallel processor allows the cost of moving information through its memory to be amortized (a). However, this comes at the cost of increasing the latency between successive outputs of the model (b). Data plotted is sourced from the public ML.Energy benchmark mlenergy-neuripsdb25.
• Figure 5: Costs to train AI models 'from scratch' scale with the number of parameters utilized in the model, and reach into billions of watt-hours (Wh). In contrast, the brain has numerous learning mechanisms available acting in tandem to enable learning at a variety of timescales with a minuscule cost when averaged over the entirety of the brain.