Multi-Dimensional Reconfigurable, Physically Composable Hybrid Diffractive Optical Neural Network

TL;DR

This work introduces, for the first time, a multi-dimensional reconfigurable hybrid diffractive ONN system (MDR-HDONN), a physically composable architecture that unlocks a new degree of freedom and unprecedented versatility in DONNs.

Abstract

Diffractive optical neural networks (DONNs), leveraging free-space light wave propagation for ultra-parallel, high-efficiency computing, have emerged as promising artificial intelligence (AI) accelerators. However, their inherent lack of reconfigurability due to fixed optical structures post-fabrication hinders practical deployment in the face of dynamic AI workloads and evolving applications. To overcome this challenge, we introduce, for the first time, a multi-dimensional reconfigurable hybrid diffractive ONN system (MDR-HDONN), a physically composable architecture that unlocks a new degree of freedom and unprecedented versatility in DONNs. By leveraging full-system learnability, MDR-HDONN repurposes fixed fabricated optical hardware, achieving exponentially expanded functionality and superior task adaptability through the differentiable learning of system variables. Furthermore, MDR-HDONN adopts a hybrid optical/photonic design, combining the reconfigurability of integrated photonics with the ultra-parallelism of free-space diffractive systems. Extensive evaluations demonstrate that MDR-HDONN has digital-comparable accuracy on various task adaptations with 74x faster speed and 194x lower energy. Compared to prior DONNs, MDR-HDONN shows exponentially larger functional space with 5x faster training speed, paving the way for a new paradigm of versatile, composable, hybrid optical/photonic AI computing. We will open-source our codes.

Figures (11)

• Figure 1: Diffractive optical neural networks (DONN) based on metasurfaces with a pixel size of $s$. The rectangular meta-atom shape can be designed to realize a polarization-dependent phase shift for orthogonally polarized light.
• Figure 2: Multi-path diffractive layer (DiffLayer) with $P$ paths. Each has $L$ cascaded metasurfaces and polarization-differential photodetection. Learnable parameters ➊-➐ are marked.
• Figure 3: 9 orientation states for each bi-directional phase mask, which can be learned differentiably using Gumbel-softmaxd.
• Figure 4: Hybrid DONNLayer with pointwise Convolution mapped to integrated PTCs and depthwise diffractive layers mapped to free-space diffractive optics.
• Figure 5: Illustration of metasurface placement order permutation. Two-stage training is adopted with warmup training and ALM-based permutation learning.