Hybrid Convolution and Frequency State Space Network for Image Compression

TL;DR

The paper tackles the efficiency gap in learned image compression by introducing HCFSSNet, a hybrid architecture that fuses a CNN branch for local high-frequency details with a Vision Frequency State Space (VFSS) branch for global, low-frequency structure. VFSS incorporates a Vision Omni-directional Neighborhood State Space (VONSS) to preserve 2D neighborhood information and an Adaptive Frequency Modulation Module (AFMM) that operates in the DCT domain, complemented by a Frequency Swin Transformer Attention Module (FSTAM) in the entropy model. Empirical results on Kodak, Tecnick, and CLIC demonstrate BD-rate reductions relative to VTM and competitive performance with fewer parameters than contemporary SSM- and Transformer-based codecs, particularly at high resolutions. The work emphasizes a compact, interpretable design that decouples local and global processing while leveraging frequency-domain modulation to improve rate–distortion efficiency. It sets a foundation for scalable, frequency-aware LIC backbones suitable for future high-capacity or real-time systems.

Abstract

Learned image compression (LIC) has recently benefited from Transformer based and state space model (SSM) based architectures. Convolutional neural networks (CNNs) effectively capture local high frequency details, whereas Transformers and SSMs provide strong long range modeling capabilities but may cause structural information loss or ignore frequency characteristics that are crucial for compression. In this work we propose HCFSSNet, a Hybrid Convolution and Frequency State Space Network for LIC. HCFSSNet uses CNNs to extract local high frequency structures and introduces a Vision Frequency State Space (VFSS) block that models long range low frequency information. The VFSS block combines an Omni directional Neighborhood State Space (VONSS) module, which scans features horizontally, vertically and diagonally, with an Adaptive Frequency Modulation Module (AFMM) that applies content adaptive weighting of discrete cosine transform frequency components for more efficient bit allocation. To further reduce redundancy in the entropy model, we integrate AFMM with a Swin Transformer to form a Frequency Swin Transformer Attention Module (FSTAM) for frequency aware side information modeling. Experiments on the Kodak, Tecnick and CLIC Professional Validation datasets show that HCFSSNet achieves competitive rate distortion performance compared with recent SSM based codecs such as MambaIC, while using significantly fewer parameters. On Kodak, Tecnick and CLIC, HCFSSNet reduces BD rate over the VTM anchor by 18.06, 24.56 and 22.44 percent, respectively, providing an efficient and interpretable hybrid architecture for future learned image compression systems.

Figures (8)

• Figure 1: Representative nonlinear transforms used in learned image compression. (a) CNN-based methods (e.g., Chen et al. chen2021nonlocal); (b) Transformer-based methods (e.g., Zhu et al. zhu2022transformerbased); (c) hybrid CNN--Transformer methods (e.g., Liu et al. liu2023learned); (d) vision state space model (SSM)–based methods (e.g., Qin et al. qin2024mambavc); and (e) the proposed HCFSSNet, which combines a CNN branch for local details with a Vision Frequency State Space (VFSS) branch for long-range dependencies.
• Figure 2: Overall architecture of the proposed HCFSSNet. The HCFSS block denotes the Hybrid Convolution-Frequency State Space block. RBS and RBU denote residual blocks for downsampling and upsampling, respectively, built from $1\times1$ and $3\times3$ convolutions. "s2" denotes a stride of 2 while "enc" and "dec" denote range encoders/decoders that integrate a quantizer. $C$ and $M$ denote the numbers of channels used for feature extraction and for latent features, respectively.
• Figure 3: (a) Illustration of the proposed HCFSS block. LReLU denotes the Leaky ReLU activation function. "Split" and "Concat" refer to channel-wise feature separation and concatenation, respectively. VFSS denotes the Vision Frequency State Space block. (b) Architecture of the VFSS block. DwConv denotes depthwise convolution. VONSSM denotes the Vision Omni-directional Neighborhood State Space Module, and AFMM stands for the Adaptive Frequency Modulation Module. (c) Architecture of the AFMM. DCT and IDCT denote the Discrete Cosine Transform and its inverse, respectively.
• Figure 4: Details of the Vision Omni-directional Neighborhood State Space Module (VONSSM), illustrating the omni-directional scanning directions.
• Figure 5: Architectural details of the proposed channel-wise entropy model. (a) Overall structure of the entropy model. (b) Slice network $e_i$. (c) Hyper-analysis and hyper-synthesis transforms. (d) Frequency Swin Transformer Attention Module (FSTAM); "SwinT block" denotes a Swin Transformer block.