Partial Channel Network: Compute Fewer, Perform Better

TL;DR

Partial Channel Mechanism (PCM) reduces computation by splitting feature channels and applying different operations in parallel, enabling Partial Attention Convolution (PATConv) and Dynamic Partial Convolution (DPConv). The PartialNet family integrates PATConv and learnable channel splits to achieve higher accuracy at lower FLOPs and higher throughput on ImageNet-1K, with strong transfer to COCO detection and segmentation. The key contributions are the PCM framework, three PATConv blocks (PAT_ch, PAT_sp, PAT_sf), the DPConv with complexity-aware training, and the four-stage PartialNet architectures that outperform SOTA on standard benchmarks. This work offers a practical path toward fast, accurate vision models suitable for real-world deployment.

Abstract

Designing a module or mechanism that enables a network to maintain low parameters and FLOPs without sacrificing accuracy and throughput remains a challenge. To address this challenge and exploit the redundancy within feature map channels, we propose a new solution: partial channel mechanism (PCM). Specifically, through the split operation, the feature map channels are divided into different parts, with each part corresponding to different operations, such as convolution, attention, pooling, and identity mapping. Based on this assumption, we introduce a novel partial attention convolution (PATConv) that can efficiently combine convolution with visual attention. Our exploration indicates that the PATConv can completely replace both the regular convolution and the regular visual attention while reducing model parameters and FLOPs. Moreover, PATConv can derive three new types of blocks: Partial Channel-Attention block (PAT_ch), Partial Spatial-Attention block (PAT_sp), and Partial Self-Attention block (PAT_sf). In addition, we propose a novel dynamic partial convolution (DPConv) that can adaptively learn the proportion of split channels in different layers to achieve better trade-offs. Building on PATConv and DPConv, we propose a new hybrid network family, named PartialNet, which achieves superior top-1 accuracy and inference speed compared to some SOTA models on ImageNet-1K classification and excels in both detection and segmentation on the COCO dataset. Our code is available at https://github.com/haiduo/PartialNet.

Figures (9)

• Figure 1: Comparison of different operation types.
• Figure 2: Our PartialNet achieves higher trade-off of accuracy and throughput on ImageNet-1K.
• Figure 3: The overall architecture of our PartialNet, consisting of four hierarchical stages, each incorporating a series of PartialNet blocks followed by an embedding or merging layer. The last three layers are dedicated to feature classification. Where $\odot$ and $\otimes$ denote element-wise multiplication and matrix multiplication respectively.
• Figure 4: The generation process of DPConv, where $\odot$ denotes elementwise product, $\circledast$ denotes a Kronecker product.
• Figure 5: The relationship between FLOPs and Top-1 Accuracy in different PartialNet model variants.