Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-inspired Global-to-Parallel Multi-scale Encoding for Lightweight Vision Models

Wei Xu

TL;DR

This work tackles the efficiency–accuracy challenge in lightweight vision models by revisiting human visual processing. It introduces Global-to-Parallel Multi-scale Encoding (GPM), where a Global Insight Generator (GIG) provides global priors that feed two parallel branches: Large-Scale Attention Encoder (LSAE) for mid-/large-scale semantics and Inverted Residual Block (IRB) with Channel Relational Attention (CRA) for fine details, further enhanced by ASA. The resulting H-GPE backbone, built from GPE-Blocks and IRB-based stages, demonstrates strong performance across image classification, object detection, and semantic segmentation while maintaining low parameter counts and FLOPs, trained on a single GPU. Key contributions include the design of GPM, the lightweight GPE-Block, and extensive multi-task evaluation showing superior accuracy–efficiency trade-offs relative to recent state-of-the-art lightweight models. The work highlights the value of human-vision-inspired multi-frequency decoupling for practical deployment on resource-constrained devices.

Abstract

Lightweight vision networks have witnessed remarkable progress in recent years, yet achieving a satisfactory balance among parameter scale, computational overhead, and task performance remains difficult. Although many existing lightweight models manage to reduce computation considerably, they often do so at the expense of a substantial increase in parameter count (e.g., LSNet, MobileMamba), which still poses obstacles for deployment on resource-limited devices. In parallel, some studies attempt to draw inspiration from human visual perception, but their modeling tends to oversimplify the visual process, making it hard to reflect how perception truly operates. Revisiting the cooperative mechanism of the human visual system, we propose GPM (Global-to-Parallel Multi-scale Encoding). GPM first employs a Global Insight Generator (GIG) to extract holistic cues, and subsequently processes features of different scales through parallel branches: LSAE emphasizes mid-/large-scale semantic relations, while IRB (Inverted Residual Block) preserves fine-grained texture information, jointly enabling coherent representation of global and local features. As such, GPM conforms to two characteristic behaviors of human vision perceiving the whole before focusing on details, and maintaining broad contextual awareness even during local attention. Built upon GPM, we further develop the lightweight H-GPE network. Experiments on image classification, object detection, and semantic segmentation show that H-GPE achieves strong performance while maintaining a balanced footprint in both FLOPs and parameters, delivering a more favorable accuracy-efficiency trade-off compared with recent state-of-the-art lightweight models.

Human-inspired Global-to-Parallel Multi-scale Encoding for Lightweight Vision Models

TL;DR

This work tackles the efficiency–accuracy challenge in lightweight vision models by revisiting human visual processing. It introduces Global-to-Parallel Multi-scale Encoding (GPM), where a Global Insight Generator (GIG) provides global priors that feed two parallel branches: Large-Scale Attention Encoder (LSAE) for mid-/large-scale semantics and Inverted Residual Block (IRB) with Channel Relational Attention (CRA) for fine details, further enhanced by ASA. The resulting H-GPE backbone, built from GPE-Blocks and IRB-based stages, demonstrates strong performance across image classification, object detection, and semantic segmentation while maintaining low parameter counts and FLOPs, trained on a single GPU. Key contributions include the design of GPM, the lightweight GPE-Block, and extensive multi-task evaluation showing superior accuracy–efficiency trade-offs relative to recent state-of-the-art lightweight models. The work highlights the value of human-vision-inspired multi-frequency decoupling for practical deployment on resource-constrained devices.

Abstract

Lightweight vision networks have witnessed remarkable progress in recent years, yet achieving a satisfactory balance among parameter scale, computational overhead, and task performance remains difficult. Although many existing lightweight models manage to reduce computation considerably, they often do so at the expense of a substantial increase in parameter count (e.g., LSNet, MobileMamba), which still poses obstacles for deployment on resource-limited devices. In parallel, some studies attempt to draw inspiration from human visual perception, but their modeling tends to oversimplify the visual process, making it hard to reflect how perception truly operates. Revisiting the cooperative mechanism of the human visual system, we propose GPM (Global-to-Parallel Multi-scale Encoding). GPM first employs a Global Insight Generator (GIG) to extract holistic cues, and subsequently processes features of different scales through parallel branches: LSAE emphasizes mid-/large-scale semantic relations, while IRB (Inverted Residual Block) preserves fine-grained texture information, jointly enabling coherent representation of global and local features. As such, GPM conforms to two characteristic behaviors of human vision perceiving the whole before focusing on details, and maintaining broad contextual awareness even during local attention. Built upon GPM, we further develop the lightweight H-GPE network. Experiments on image classification, object detection, and semantic segmentation show that H-GPE achieves strong performance while maintaining a balanced footprint in both FLOPs and parameters, delivering a more favorable accuracy-efficiency trade-off compared with recent state-of-the-art lightweight models.
Paper Structure (22 sections, 3 equations, 5 figures, 9 tables)

This paper contains 22 sections, 3 equations, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Attention Mechanisms
  4. MHSA and Window-based MHSA
  5. Convolutional Attention Mechanisms
  6. Hybrid Transformer–CNN Models
  7. The proposed method
  8. GPM Architecture
  9. GIG
  10. LSAE and ASA
  11. IRB and CRA
  12. Summary of GPM
  13. GPE-Block
  14. H-GPE Model
  15. Stem and Downsample Layers
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Human visual processing pipeline and the recognition workflow of H-GPE.
  • Figure 2: Comparison of accuracy, parameter counts, and FLOPs on the ImageNet validation set, where the circle size encodes model complexity. H-GPE demonstrates strong competitiveness.
  • Figure 3: (a) GIG module; (b) LSAE module; (c) CRA module; (d) ASA module; and (e) GPM architecture. $SP\_H$ and $SP\_W$ indicate strip pooling along the height and width directions, respectively; $conv$ represents convolution; and Window Partition/Merging denote the splitting and merging of feature maps.
  • Figure 4: IRB and GPM-Block,$BN$ denotes Batch Normalization
  • Figure 5: Overview of the H-GPE architecture