What Is Next for LLMs? Next-Generation AI Computing Hardware Using Photonic Chips

TL;DR

Large language models require immense compute and energy, prompting exploration of photonic and spintronic hardware as non-von Neumann alternatives. The paper surveys photonic neural-network components (Microring resonators, Mach-Zehnder interferometers, metasurfaces, lasers), 2D-material on-chip integration (graphene, TMDCs), spintronic neuromorphic devices, transformer-focused hardware mappings, and spiking neural networks, outlining methods to realize high-throughput matrix ops and attention with lower energy. It highlights potential order-of-magnitude gains in throughput and efficiency but warns of critical bottlenecks in long-context memory, mega-scale data storage, precision, and nonlinear activation on photonic substrates, calling for hardware-software co-design and memory-centric architectures. The significance lies in charting a path toward exascale-like AI hardware through integrated photonic/spintronic platforms, system co-design, and novel memory/storage solutions, while acknowledging substantial open challenges before photonic LLMs can rival electronic counterparts at scale.

Abstract

Large language models (LLMs) are rapidly pushing the limits of contemporary computing hardware. For example, training GPT-3 has been estimated to consume around 1300 MWh of electricity, and projections suggest future models may require city-scale (gigawatt) power budgets. These demands motivate exploration of computing paradigms beyond conventional von Neumann architectures. This review surveys emerging photonic hardware optimized for next-generation generative AI computing. We discuss integrated photonic neural network architectures (e.g., Mach-Zehnder interferometer meshes, lasers, wavelength-multiplexed microring resonators) that perform ultrafast matrix operations. We also examine promising alternative neuromorphic devices, including spiking neural network circuits and hybrid spintronic-photonic synapses, which combine memory and processing. The integration of two-dimensional materials (graphene, TMDCs) into silicon photonic platforms is reviewed for tunable modulators and on-chip synaptic elements. Transformer-based LLM architectures (self-attention and feed-forward layers) are analyzed in this context, identifying strategies and challenges for mapping dynamic matrix multiplications onto these novel hardware substrates. We then dissect the mechanisms of mainstream LLMs, such as ChatGPT, DeepSeek, and LLaMA, highlighting their architectural similarities and differences. We synthesize state-of-the-art components, algorithms, and integration methods, highlighting key advances and open issues in scaling such systems to mega-sized LLM models. We find that photonic computing systems could potentially surpass electronic processors by orders of magnitude in throughput and energy efficiency, but require breakthroughs in memory, especially for long-context windows and long token sequences, and in storage of ultra-large datasets.

Figures (22)

• Figure 1: Microring resonator: a Neuromorphic ONNs can be realized through microring resonator (MRR) weight banks. tait2017neuromorphic b The underlying mechanism and experimental setup of fully optical spiking neural networks are illustrated in feldmann2019all. c A photonic convolution accelerator has been developed using a time-wavelength multiplexing approach. xu202111 d In-memory photonic computing architectures leverage on-chip microcombs and phase-change materials. feldmann2021parallel e Microcomb-based integrated ONNs enable convolution operations for applications such as emotion recognition. cheng2023human
• Figure 2: Mach-Zehnder Interferometer: a Training methodology diagram for ONNs enabling real-time in-situ learning hughes2018training b Integrated photonic neural network architecture combining MZIs with diffractive optical components zhu2022space c Demonstrated in situ backpropagation training of a photonic neural network using MZI meshes. pai2023experimentally
• Figure 3: 2D Metasurface: a Conceptual representation of the inference mechanism in diffractive deep neural networks (D2NN). lin2018all b Experimental configuration demonstrating logical operations through diffractive optical neural networks (DONN). qian2020performing c Nanoprinted optical perceptrons enable on-chip. goi2021nanoprinted d Reconfigurable DONN architecture utilizing digital meta-atom arrays. liu2022programmable
• Figure 4: 1D Metasurface: a Experimental validation of 1D DONNs for photonic machine learning. wang2019chip b Simulation-based validation of on-chip DONN with light-speed computation. zarei2020integrated c Dielectric metasurface enables on-chip wavefront control for Fourier transform and spatial differentiation. fu2023photonic
• Figure 5: 4f system: a A hybrid optoelectronic CNN using 4f optical setup. chang2018hybrid b An entirely ONN architecture where a deep diffractive neural network is integrated into the Fourier plane of a 4f imaging system. yan2019fourier