Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VisionGRU: A Linear-Complexity RNN Model for Efficient Image Analysis

Shicheng Yin, Kaixuan Yin, Weixing Chen, Enbo Huang, Yang Liu

TL;DR

VisionGRU tackles the high computational cost of CNNs and ViTs in high-resolution image analysis by introducing a linear-complexity RNN backbone built on minGRU units. It employs a hierarchical 2DGRU module with bidirectional scanning to capture local and global context in 2D feature maps, enabling efficient multi-scale representation. Empirical results on ImageNet and ADE20K show VisionGRU achieving state-of-the-art-like accuracy with substantially lower FLOPs and memory use compared to ViTs, while maintaining competitive segmentation performance. The work demonstrates that carefully designed recurrent architectures can rival transformer-based approaches in efficiency and scalability for computer vision tasks.

Abstract

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) are two dominant models for image analysis. While CNNs excel at extracting multi-scale features and ViTs effectively capture global dependencies, both suffer from high computational costs, particularly when processing high-resolution images. Recently, state-space models (SSMs) and recurrent neural networks (RNNs) have attracted attention due to their efficiency. However, their performance in image classification tasks remains limited. To address these challenges, this paper introduces VisionGRU, a novel RNN-based architecture designed for efficient image classification. VisionGRU leverages a simplified Gated Recurrent Unit (minGRU) to process large-scale image features with linear complexity. It divides images into smaller patches and progressively reduces the sequence length while increasing the channel depth, thus facilitating multi-scale feature extraction. A hierarchical 2DGRU module with bidirectional scanning captures both local and global contexts, improving long-range dependency modeling, particularly for tasks like semantic segmentation. Experimental results on the ImageNet and ADE20K datasets demonstrate that VisionGRU outperforms ViTs, significantly reducing memory usage and computational costs, especially for high-resolution images. These findings underscore the potential of RNN-based approaches for developing efficient and scalable computer vision solutions. Codes will be available at https://github.com/YangLiu9208/VisionGRU.

VisionGRU: A Linear-Complexity RNN Model for Efficient Image Analysis

TL;DR

VisionGRU tackles the high computational cost of CNNs and ViTs in high-resolution image analysis by introducing a linear-complexity RNN backbone built on minGRU units. It employs a hierarchical 2DGRU module with bidirectional scanning to capture local and global context in 2D feature maps, enabling efficient multi-scale representation. Empirical results on ImageNet and ADE20K show VisionGRU achieving state-of-the-art-like accuracy with substantially lower FLOPs and memory use compared to ViTs, while maintaining competitive segmentation performance. The work demonstrates that carefully designed recurrent architectures can rival transformer-based approaches in efficiency and scalability for computer vision tasks.

Abstract

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) are two dominant models for image analysis. While CNNs excel at extracting multi-scale features and ViTs effectively capture global dependencies, both suffer from high computational costs, particularly when processing high-resolution images. Recently, state-space models (SSMs) and recurrent neural networks (RNNs) have attracted attention due to their efficiency. However, their performance in image classification tasks remains limited. To address these challenges, this paper introduces VisionGRU, a novel RNN-based architecture designed for efficient image classification. VisionGRU leverages a simplified Gated Recurrent Unit (minGRU) to process large-scale image features with linear complexity. It divides images into smaller patches and progressively reduces the sequence length while increasing the channel depth, thus facilitating multi-scale feature extraction. A hierarchical 2DGRU module with bidirectional scanning captures both local and global contexts, improving long-range dependency modeling, particularly for tasks like semantic segmentation. Experimental results on the ImageNet and ADE20K datasets demonstrate that VisionGRU outperforms ViTs, significantly reducing memory usage and computational costs, especially for high-resolution images. These findings underscore the potential of RNN-based approaches for developing efficient and scalable computer vision solutions. Codes will be available at https://github.com/YangLiu9208/VisionGRU.

Paper Structure

This paper contains 20 sections, 9 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Convolutional Neural Networks (CNNs) and Their Variants
  4. Models Based on Self-Attention Mechanisms
  5. Vision Models Based on State Space Models (SSM)
  6. Traditional RNNs and Their Limitations
  7. Efficiently Trainable RNN Variants
  8. Methodology
  9. Preliminary
  10. RNN Theory
  11. GRU Unit
  12. minGRU Unit
  13. VisionGRU Architecture
  14. 2DGRU Module
  15. Details
  16. ...and 5 more sections

Figures (3)

  • Figure 1: VisionGRU-Ti achieves higher classification accuracy (82%) and semantic segmentation mIoU (44.7%) compared to DeiT-S models. Moreover, VisionGRU-Ti exhibits significantly lower FLOPs at all resolutions, requiring 151.9 GFLOPs at 1248×1248 compared to DeiT-S's 432.3 GFLOPs, highlighting its computational efficiency.
  • Figure 2: Overview of the VisionGRU model. It integrates the computational strengths of RNNs and CNNs using a hierarchical 2DGRU module with a bidirectional scanning strategy for efficient global feature capture.
  • Figure 3: The semantic segmentation example from the ADE20K validation set. The left image shows the segmentation result of Swin, while the right image shows the result of our VisionGRU.