SpiralMLP: A Lightweight Vision MLP Architecture

TL;DR

SpiralMLP introduces Spiral FC, a spiral-offset, deformable-convolution-inspired token-mixing layer that replaces traditional Token Mixing in MLP-based vision models. By combining Self-Spiral FC and Cross-Spiral FC through a Learnable Merge Head, Spiral Mixing captures diverse spatial information with linear complexity $O(HW)$ and is compatible with PVT-style and Swin-style backbones. Across ImageNet-1k, COCO, and ADE20K, SpiralMLP delivers competitive or superior accuracy while using fewer parameters and lower FLOPs than many Transformer- and CNN-based SOTAs, as shown by strong Top-1, AP, and mIoU results and supported by ablation studies on offset design, partitioning $k$, and amplitude $A_{ ext{max}}$. The work demonstrates that a refined spiral sampling strategy can effectively cover the full receptive field without a heavy computational burden, offering a practical path toward efficient, high-performance vision models.

Abstract

We present SpiralMLP, a novel architecture that introduces a Spiral FC layer as a replacement for the conventional Token Mixing approach. Differing from several existing MLP-based models that primarily emphasize axes, our Spiral FC layer is designed as a deformable convolution layer with spiral-like offsets. We further adapt Spiral FC into two variants: Self-Spiral FC and Cross-Spiral FC, which enable both local and global feature integration seamlessly, eliminating the need for additional processing steps. To thoroughly investigate the effectiveness of the spiral-like offsets and validate our design, we conduct ablation studies and explore optimal configurations. In empirical tests, SpiralMLP reaches state-of-the-art performance, similar to Transformers, CNNs, and other MLPs, benchmarking on ImageNet-1k, COCO and ADE20K. SpiralMLP still maintains linear computational complexity O(HW) and is compatible with varying input image resolutions. Our study reveals that targeting the full receptive field is not essential for achieving high performance, instead, adopting a refined approach offers better results.

Figures (8)

• Figure 1: (a) While the Channel FC concentrates solely at the target point, marked with a $\bigstar$, the Spiral FC captures richer spatial information. Spiral FC is in accordance with \ref{['eq:6', 'eq:7']}, the input channel dimension $C_{\text{in}}=14$, the maximum amplitude $A_{\text{max}}=6$ and $T=8$. The coordinate numbers are arranged as $(H, W, C)$. This illustrative example only contains half of the $C_{\text{in}}$. (b) provides a complete visualization when the parameters are: $C_{\text{in}}=20$, $A_{\text{max}}=3$ and $T = 8$.
• Figure 2: (a) displays the comprehensive architecture of SpiralMLP in PVT-style, featuring four distinct stages. Each stage is composed of multiple Patch Embedding layers and identically-configured Spiral Blocks. (b) explores the internal layout of a Spiral Block, where the proposed Spiral Mixing replaces the traditional Token Mixing. (c) outlines the components of Spiral Mixing, which incorporates the meticulously designed Spiral FC to effectively capture spatial information.
• Figure 3: In contrast to ASMLP lian2022asmlp and Swin liu2021swin, our SpiralMLP demonstrates superior object-focused attention. SpiralMLP exhibits enhanced sensitivity, especially for elongated or curved objects. The backbones employed for heatmaps generation are SpiralMLP-B5, ASMLP-B and Swin-B. The images are sourced from the ImageNet-1k russakovsky2015imagenet validation dataset, with corresponding labels.
• Figure 4: Several examples of the object detection and instance segmentation from COCO lin2015microsoft test dataset.
• Figure 5: Several examples of the semantic segmentation from ADE20K zhou2018semantic validation dataset.