Programmable metasurfaces for future photonic artificial intelligence

Abstract

Photonic neural networks (PNNs), which share the inherent benefits of photonic systems, such as high parallelism and low power consumption, could challenge traditional digital neural networks in terms of energy efficiency, latency, and throughput. However, producing scalable photonic artificial intelligence (AI) solutions remains challenging. To make photonic AI models viable, the scalability problem needs to be solved. Large optical AI models implemented on PNNs are only commercially feasible if the advantages of optical computation outweigh the cost of their input-output overhead. In this Perspective, we discuss how field-programmable metasurface technology may become a key hardware ingredient in achieving scalable photonic AI accelerators and how it can compete with current digital electronic technologies. Programmability or reconfigurability is a pivotal component for PNN hardware, enabling in situ training and accommodating non-stationary use cases that require fine-tuning or transfer learning. Co-integration with electronics, 3D stacking, and large-scale manufacturing of metasurfaces would significantly improve PNN scalability and functionalities. Programmable metasurfaces could address some of the current challenges that PNNs face and enable next-generation photonic AI technology.

Figures (6)

• Figure 1: An illustration of the perspective programmable metasurface technology for optical computation. The electrically connected programmable metasurface (panel c) consists of subwavelength voltage-actuated pixels (panel d). Due to the metasurface’s reconfigurability, structural nonlinearities can be implemented. Such technology could mimic electronic processors, such as a graphics processing unit (GPU, panel a), which consists of billions of transistor nodes (panel b).
• Figure 2: Illustration of training schemes used in free-space photonic neural networks.a, In silico training performed on a digital twin of the physical diffractive neural network that is trained via error back-propagation. b, Physics-aware training (PAT) in which the forward pass is performed on the physical neural network, whose output is then used to compute the loss function and the back-propagated gradients in a digital model (denoted by $f_{\mathrm{model}}$), which are then used to update the physical system with the optimized parameters. c, Forward-forward PAT in which a local 'goodness' loss function is computed and minimized at the output of each layer following two forward passes, one in which positive data are input with correct labels and another in which negative data are input with incorrect labels. d, PAT with an optoelectronic loop in which a spatial light modulator (SLM) modulates the input from a digital micro-mirror device (DMD) in phase. A sensor reads out the output of the SLM, which is then fed back as an input for the DMD. Reiterating this feedback process several times mimics a multilayer diffractive neural network. The physical output at each layer is used to optimize the phase values of the SLM via back-propagation on a digital model. It should be noted here that this method can be used with any phase-modulating element other than an SLM. All experimental results shown are for a digit classification inference task. Panel a adapted from lin2018all, panel b adapted fromwright2022deep, panel c adapted from momeni2023backpropagation, panel d adapted from zhou2021large.
• Figure 3: Physical mechanisms used to reconfigure electromagnetic fields in the radio-frequency and optical domains.a, Radio-frequency reconfigurable metasurfaces rely on the modulation of the reflectivity of an array of sub-wavelength patch antennas made from a conducting material. Each antenna is connected to a lumped impedance (either a PIN diode or a varactor diode) that can be electrically modulated between a low-resistance state (impedance A, modelled primarily by an RL circuit) and a high-resistance or capacitive element (impedance B, modelled by a CL circuit). This modifies the antenna's impedance and modulates the reflection of electromagnetic waves from the antenna. sievenpiper2003twokamoda201160b, The electro-optic Pockels effectkarvounis2020electrobenea2021electroweigand2021enhancedzheng2024dynamic can induce an electrically modulated refractive index in a material consisting of non-centrosymmetric molecules (represented by teardrop-shaped dipoles) to tune light waves. c, Liquid-crystal molecules in a unit cell consisting of two parallel electrodes across which an electric field is applied reorient themselves such that they are aligned along the field lines. This results in a modulation of the refractive index of the liquid-crystal from the ordinary ($n_{\text{o}}$) to the extraordinary ($n_{\text{e}}$) refractive index, effectively modulating electromagnetic waves traversing the liquid-crystal unit cell. komar2017electricallykomar2018dynamickowerdziej2019ultrafastsun2019efficientli2019phasedolan2021broadbandd, Phase-change materials can be switched from an amorphous to a crystalline state (and vice versa) through the application of a stimulus, such as the modulation of the material's temperature, thus modulating the material's refractive indexhuang2016gatenicholls2017ultrafastlewi2017ultrawidewang2021electricalzhang2021electricallycueff2021reconfigurableking2024electrically.
• Figure 4: Possible neural network architectures with programmable metasurfaces. Potential neural networks implemented with programmable metasurface matrix-vector multipliers (MVMs), which rely on sub-wavelength pixels that can be actuated either with a DC or an AC electrical bias, enable a, wavelength multiplexing and b, multitasking. The multitasking capability arises from the AC modulation that induces signal harmonics, each of which is used to encode a different output. c, A schematic illustration of a programmable metasurface MVM front-end-integrated and d, co-integrated as an optical AI accelerator to a digital neural network. e, Schematic of a stacked programmable metasurface MVM neural network subjected to structural nonlinearity by superposing the input information at each layer.
• Figure :