Deep Linear Array Pushbroom Image Restoration: A Degradation Pipeline and Jitter-Aware Restoration Network

TL;DR

The paper tackles distortion and blur in linear array pushbroom (LAP) remote-sensing images caused by camera jitter. It introduces a Continue Dynamic Shooting Model (CDSM)-based jitter simulation to generate a realistic degradation pipeline and a jitter-aware two-stage restoration network, JARNet, consisting of an Optical Flow Correction (OFC) stage and a Spatial and Frequency Residual (SFRes) stage with a Coordinate Attention and a Frequency Branch. The approach leverages jitter priors in both spatial and frequency domains, yielding state-of-the-art restoration results (about $+1.28$ dB PSNR over strong baselines) on a LAP dataset synthesized from DOTA-v1.0 and demonstrating good real-world performance and cross-dataset generalization. This work provides a scalable framework for LAP image restoration by combining principled jitter modeling with architecture-level jitter awareness, and it includes an end-to-end data synthesis pipeline to address limited real LAP data.

Abstract

Linear Array Pushbroom (LAP) imaging technology is widely used in the realm of remote sensing. However, images acquired through LAP always suffer from distortion and blur because of camera jitter. Traditional methods for restoring LAP images, such as algorithms estimating the point spread function (PSF), exhibit limited performance. To tackle this issue, we propose a Jitter-Aware Restoration Network (JARNet), to remove the distortion and blur in two stages. In the first stage, we formulate an Optical Flow Correction (OFC) block to refine the optical flow of the degraded LAP images, resulting in pre-corrected images where most of the distortions are alleviated. In the second stage, for further enhancement of the pre-corrected images, we integrate two jitter-aware techniques within the Spatial and Frequency Residual (SFRes) block: 1) introducing Coordinate Attention (CoA) to the SFRes block in order to capture the jitter state in orthogonal direction; 2) manipulating image features in both spatial and frequency domains to leverage local and global priors. Additionally, we develop a data synthesis pipeline, which applies Continue Dynamic Shooting Model (CDSM) to simulate realistic degradation in LAP images. Both the proposed JARNet and LAP image synthesis pipeline establish a foundation for addressing this intricate challenge. Extensive experiments demonstrate that the proposed two-stage method outperforms state-of-the-art image restoration models. Code is available at https://github.com/JHW2000/JARNet.

Figures (11)

• Figure 1: Illustration of our main idea: The stitched LAP image suffers from distortion and blur caused by camera jitter. To restore degraded LAP image, we propose a jitter pre-correction and enhancement method in a two-stage manner. We also design an image synthesis pipeline for training data acquisition. Finally, our proposed JARNet outperforms state-of-the-art methods on our LAP dataset.
• Figure 2: Overall degradation and restoration pipeline for LAP images. (a) Proposed LAP image degradation pipeline. (b) Proposed CDSM-based jitter map generating procedure. (c) Proposed two-stage restoration pipeline for LAP images. (d) Visualization of jitter curve in (b). From upper left to lower right: ideal jitter curve, the average of finer-grained ideal jitter samples from CDSM, noisy jitter curve, and the average of finer-grained noisy jitter samples from CDSM. For simplicity, we only display jitter curve in the roll direction. (e) Visualization of optical flow from noisy jitter map.
• Figure 3: Examples of our LAP image dataset. The $1^{st}$ and $3^{rd}$ columns are simulated LAP images from our proposed degradation pipeline. The $2^{nd}$ and $4^{th}$ columns are the corresponding original clean scene.
• Figure 4: Architecture of JARNet for LAP image restoration. (a) Overview of our two-stage JARNet. (b) Details of SFRes block. 'Norm' means layer normalization. 'Dconv' means depthwise convolution. 'SG' means SimpleGate chen2022simple. 'CoA' means coordinate attention hou2021coordinate. 'Freq Branch' means frequency branch mao2023intriguing.
• Figure 5: Details of our proposed Optical Flow Correction (OFC) block. 'Tanh' means hyperbolic tangent activation function. 'SCA' means Simplified Channel Attention chen2022simple.