Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Network Methods for Radiation Detectors and Imaging

S. Lin, S. Ning, H. Zhu, T. Zhou, C. L. Morris, S. Clayton, M. Cherukara, R. T. Chen, Z. Wang

TL;DR

The paper surveys how deep learning and neural networks are transforming radiation detectors and imaging by addressing the data deluge from high-brightness photon sources, enabling real-time edge processing, and reducing data movement through hardware acceleration. It surveys data generation at photon sources, DL-based image processing methods (restoration, segmentation, compression, sparse sampling, 3D reconstruction), and a spectrum of hardware solutions from CPUs/GPUs and FPGAs/ASICs to optical neural networks and spiking neuromorphic hardware. The key contributions include concrete DL approaches applied to tomography and X-ray imaging (e.g., TomoGAN, Sparse2Noise, PtychoNN, AutoPhaseNN) and a comprehensive review of edge-centric hardware trends, highlighting trade-offs between programmability, latency, and energy efficiency. The findings emphasize that combining DL with edge hardware—especially analog neuromorphic and photonic approaches—offers the most promising path to real-time, low-power analysis in photon science while alleviating data transfer and storage bottlenecks.

Abstract

Recent advances in image data processing through machine learning and especially deep neural networks (DNNs) allow for new optimization and performance-enhancement schemes for radiation detectors and imaging hardware through data-endowed artificial intelligence. We give an overview of data generation at photon sources, deep learning-based methods for image processing tasks, and hardware solutions for deep learning acceleration. Most existing deep learning approaches are trained offline, typically using large amounts of computational resources. However, once trained, DNNs can achieve fast inference speeds and can be deployed to edge devices. A new trend is edge computing with less energy consumption (hundreds of watts or less) and real-time analysis potential. While popularly used for edge computing, electronic-based hardware accelerators ranging from general purpose processors such as central processing units (CPUs) to application-specific integrated circuits (ASICs) are constantly reaching performance limits in latency, energy consumption, and other physical constraints. These limits give rise to next-generation analog neuromorhpic hardware platforms, such as optical neural networks (ONNs), for high parallel, low latency, and low energy computing to boost deep learning acceleration.

Neural Network Methods for Radiation Detectors and Imaging

TL;DR

The paper surveys how deep learning and neural networks are transforming radiation detectors and imaging by addressing the data deluge from high-brightness photon sources, enabling real-time edge processing, and reducing data movement through hardware acceleration. It surveys data generation at photon sources, DL-based image processing methods (restoration, segmentation, compression, sparse sampling, 3D reconstruction), and a spectrum of hardware solutions from CPUs/GPUs and FPGAs/ASICs to optical neural networks and spiking neuromorphic hardware. The key contributions include concrete DL approaches applied to tomography and X-ray imaging (e.g., TomoGAN, Sparse2Noise, PtychoNN, AutoPhaseNN) and a comprehensive review of edge-centric hardware trends, highlighting trade-offs between programmability, latency, and energy efficiency. The findings emphasize that combining DL with edge hardware—especially analog neuromorphic and photonic approaches—offers the most promising path to real-time, low-power analysis in photon science while alleviating data transfer and storage bottlenecks.

Abstract

Recent advances in image data processing through machine learning and especially deep neural networks (DNNs) allow for new optimization and performance-enhancement schemes for radiation detectors and imaging hardware through data-endowed artificial intelligence. We give an overview of data generation at photon sources, deep learning-based methods for image processing tasks, and hardware solutions for deep learning acceleration. Most existing deep learning approaches are trained offline, typically using large amounts of computational resources. However, once trained, DNNs can achieve fast inference speeds and can be deployed to edge devices. A new trend is edge computing with less energy consumption (hundreds of watts or less) and real-time analysis potential. While popularly used for edge computing, electronic-based hardware accelerators ranging from general purpose processors such as central processing units (CPUs) to application-specific integrated circuits (ASICs) are constantly reaching performance limits in latency, energy consumption, and other physical constraints. These limits give rise to next-generation analog neuromorhpic hardware platforms, such as optical neural networks (ONNs), for high parallel, low latency, and low energy computing to boost deep learning acceleration.
Paper Structure (25 sections, 11 figures, 1 table)

This paper contains 25 sections, 11 figures, 1 table.

Table of Contents

  1. Introduction
  2. Experimental data generation at photon sources
  3. Radiation Detectors and Imaging for Photon Science
  4. Imaging Modalities
  5. Real Time In-pixel Data-processing
  6. Deep Learning for Image Processing
  7. Neural Network Architectures
  8. Convolutional Neural Networks (CNNs)
  9. Long-Short Term Memory (LSTM)
  10. Encoder-Decoders
  11. Generative Adversarial Networks (GANs)
  12. Image Processing Techniques
  13. Restoration
  14. Segmentation
  15. Compression
  16. ...and 10 more sections

Figures (11)

  • Figure 1: Evolution of digital image sensor technology, which started with the introduction of the charge-coupled device (CCD) in the late 1960s. The latest trend is smart multi-functional CMOS image sensors enabled by three-dimensional (3D) integration in fabrication, innovations in heterogeneous materials and structures, neural networks, and edge computing. The upper left plot shows the increasing growth of sales for CMOS image sensors from 2016 to a projected value in 2026 ICinsights.
  • Figure 2: Integrated neural network and image sensor for real-time data processing and reductions.
  • Figure 3: Basic neural network architectures for (a) CNNs, (b) LSTMs, (c) encoder-decoders, and (d) GANs.
  • Figure 4: Sparse-sampled single-shot ptychography reconstruction using PtychoNN. (a) scanning probe positions with minimal overlap. (b) Single-shot PtychoNN predictions on 25$\times$ sub-sampled data compared to (c) ePIE reconstruction of the full resolution dataset.
  • Figure 5: FAST framework for autonomous experimentation. Top panel shows the workflow that enables real-time steering of scanning microscopy experiments. Bottom panel shows reconstructed images at 5%, 15% and 20% sampling along with the corresponding locations from which they were sampled. In addition, the full-grid pointwise scan and corresponding points sampled between 15-20% is also shown. Reproduced with permission from kandel2023demonstration.
  • ...and 6 more figures