FFCA-Net: Stereo Image Compression via Fast Cascade Alignment of Side Information

TL;DR

FFCA-Net addresses the latency and efficiency gap in stereo image compression by exploiting side information available at the decoder through a coarse-to-fine cascade: feature-domain stereo patch matching guided by stereo priors, a pyramid hourglass-based sparse stereo refinement, and a fast feature fusion decoder. The method jointly optimizes rate-distortion with inter-view feature alignment, achieving substantial bitrate savings (BD-rate reductions up to tens of percent across MS-SSIM and PSNR metrics) while delivering 3–10x faster decoding than prior learning-based SIC approaches. Experimental results on KITTI-stereo, Cityscapes, and InStereo2K demonstrate state-of-the-art performance, with significantly lower FLOPs and latency, and ablation studies confirm the critical contributions of patch matching, sparse refinement, and FFF. These results indicate strong practical impact for real-time, decoder-side SIC in automotive and AR/VR contexts, where low latency and bandwidth efficiency are essential.

Abstract

Multi-view compression technology, especially Stereo Image Compression (SIC), plays a crucial role in car-mounted cameras and 3D-related applications. Interestingly, the Distributed Source Coding (DSC) theory suggests that efficient data compression of correlated sources can be achieved through independent encoding and joint decoding. This motivates the rapidly developed deep-distributed SIC methods in recent years. However, these approaches neglect the unique characteristics of stereo-imaging tasks and incur high decoding latency. To address this limitation, we propose a Feature-based Fast Cascade Alignment network (FFCA-Net) to fully leverage the side information on the decoder. FFCA adopts a coarse-to-fine cascaded alignment approach. In the initial stage, FFCA utilizes a feature domain patch-matching module based on stereo priors. This module reduces redundancy in the search space of trivial matching methods and further mitigates the introduction of noise. In the subsequent stage, we utilize an hourglass-based sparse stereo refinement network to further align inter-image features with a reduced computational cost. Furthermore, we have devised a lightweight yet high-performance feature fusion network, called a Fast Feature Fusion network (FFF), to decode the aligned features. Experimental results on InStereo2K, KITTI, and Cityscapes datasets demonstrate the significant superiority of our approach over traditional and learning-based SIC methods. In particular, our approach achieves significant gains in terms of 3 to 10-fold faster decoding speed than other methods.

Figures (6)

• Figure 1: Overview of various structures for stereo image coding, including (a) joint encoding architecture and (b) asymmetric DSC structure. (c) briefly outlines the coarse-to-fine alignment method employed in our proposed FFCA-Net.
• Figure 2: The overview of the proposed model architecture. ENC and DEC refer to the encoder and decoder of the baseline single-image compressor, respectively. FEN represents the feature extraction network used to extract precise side information features.
• Figure 3: Different match results.
• Figure 4: One iteration of fast feature fusion network.
• Figure 5: Rate–distortion curves for PSNR (dB) and MS-SSIM with various compression methods.