Illumination Angular Spectrum Encoding for Controlling the Functionality of Diffractive Networks

TL;DR

This paper tackles multi-task functionality in all-optical diffractive networks by introducing illumination angular spectrum encoding, which uses amplitude masks and a 2f relay to imprint task-specific angular components onto the illumination. The same network can then produce different outputs for the same input based on the illumination pattern, enabling image-to-image translation and multi-dataset classification without altering the diffractive layers. The authors compare angular spectrum encoding to wavelength encoding, show benefits of combining the approaches, and demonstrate robustness under broadband and spatially incoherent illumination, highlighting its scalability as a degree of freedom for scalable, multi-functional optical computing. Overall, the method offers a general, compatible, and potentially practical path to multi-task all-optical computing with diffractive networks.

Abstract

Diffractive neural networks have recently emerged as a promising framework for all-optical computing. However, these networks are typically trained for a single task, limiting their potential adoption in systems requiring multiple functionalities. Existing approaches to achieving multi-task functionality either modify the mechanical configuration of the network per task or use a different illumination wavelength or polarization state for each task. In this work, we propose a new control mechanism, which is based on the illumination's angular spectrum. Specifically, we shape the illumination using an amplitude mask that selectively controls its angular spectrum. We employ different illumination masks for achieving different network functionalities, so that the mask serves as a unique task encoder. Interestingly, we show that effective control can be achieved over a very narrow angular range, within the paraxial regime. We numerically illustrate the proposed approach by training a single diffractive network to perform multiple image-to-image translation tasks. In particular, we demonstrate translating handwritten digits into typeset digits of different values, and translating handwritten English letters into typeset numbers and typeset Greek letters, where the type of the output is determined by the illumination's angular components. As we show, the proposed framework can work under different coherence conditions, and can be combined with existing control strategies, such as different wavelengths. Our results establish the illumination angular spectrum as a powerful degree of freedom for controlling diffractive networks, enabling a scalable and versatile framework for multi-task all-optical computing.

Figures (19)

• Figure 1: Achieving multiple functionalities through illumination angular spectrum encoding. (a) Schematic illustration of the proposed illumination angular spectrum encoding. An illumination mask is applied to a plane wave to retain only specific angular components. A $2f$ system maps the resulting field into its Fourier transform, creating an illumination profile that encodes a control signal for the diffractive network. The resulting profile then illuminates the object and propagates through the diffractive layers. Each illumination mask corresponds to a different task, enabling a single network to perform multiple functionalities. (b,c) Qualitative results for a single network trained with multiple predetermined ring-shaped illumination masks. In (b), a handwritten letter is transformed to either a digit or a lowercase Greek letter or an uppercase Greek letter, according to which one of three possible illumination masks is used. Similarly, in (c), a handwritten digit is transformed into one of four typeset digits depending on the value of the input digit and the mask used.
• Figure 2: Evaluation of angular-spectrum-encoded handwritten digit translation with different illumination masks. (a) An input image of a handwritten '5' is translated into different typeset digits, with the target digit selected by the angular components of the incident illumination. Rows 1, 3, and 5 show the illumination masks (Rings, Squares, and Learned), while rows 2, 4, and 6 show examples of the corresponding networks' output. The seventh row, provided for comparison, shows outputs from four specialized networks, each trained using all illumination angular components but optimized to perform a single translation task. (b) PSNR and (c) SSIM averaged over the test set for the different networks. Performance of the specialized networks is shown in the rightmost column for reference.
• Figure 3: Evaluation of angular-spectrum-encoded handwritten digit translation with different illumination masks for eight tasks. (a) An input image of a handwritten '6' is translated into different typeset digits, Greek and English letters by a single network. The target output is determined by the angular components of the illumination. Rows 1, 3, and 5 show the illumination masks (Rings, Rectangles and Learned), while rows 2, 4, and 6 show examples of the corresponding networks' output. (b) PSNR and (c) SSIM averaged over the test set for the different networks.
• Figure 4: Combining angular spectrum encoding and wavelength encoding. (a) An input image of a handwritten '4' is translated into different typeset digits. Rows 1 and 3 show different illumination masks (Rings and Learned), while rows 2 and 4 show the corresponding outputs of a single network trained with each mask type and a distinct wavelength, indicated by the color of the illumination mask (400 or 700 nm, detailed above each image). The fifth row illustrates the results of a network trained with wavelength encoding, the used wavelength for each task is indicated above each image. (b) PSNR and (c) SSIM averaged over the test set for the different networks.
• Figure 5: Angular spectrum encoding with different illumination conditions. An input image of a handwritten '7' is translated into different typeset digits. The second row illustrates the results of a network trained with spatially coherent broadband illumination. Note the speckle patterns visible in the background. The third row illustrates the results of a network trained with spatially incoherent monochromatic illumination.