Vision Mamba Distillation for Low-resolution Fine-grained Image Classification

TL;DR

A lightweight super-resolution vision Mamba classification network (SRVM-Net) is proposed to improve its capability for extracting visual features by redesigning the classification sub-network with Mamba modeling, and a novel multi-level Mamba knowledge distillation loss boosting the performance is designed.

Abstract

Low-resolution fine-grained image classification has recently made significant progress, largely thanks to the super-resolution techniques and knowledge distillation methods. However, these approaches lead to an exponential increase in the number of parameters and computational complexity of models. In order to solve this problem, in this letter, we propose a Vision Mamba Distillation (ViMD) approach to enhance the effectiveness and efficiency of low-resolution fine-grained image classification. Concretely, a lightweight super-resolution vision Mamba classification network (SRVM-Net) is proposed to improve its capability for extracting visual features by redesigning the classification sub-network with Mamba modeling. Moreover, we design a novel multi-level Mamba knowledge distillation loss boosting the performance, which can transfer prior knowledge obtained from a High-resolution Vision Mamba classification Network (HRVM-Net) as a teacher into the proposed SRVM-Net as a student. Extensive experiments on seven public fine-grained classification datasets related to benchmarks confirm our ViMD achieves a new state-of-the-art performance. While having higher accuracy, ViMD outperforms similar methods with fewer parameters and FLOPs, which is more suitable for embedded device applications. Code is available at https://github.com/boa2004plaust/ViMD.

Figures (3)

• Figure 1: Effectiveness and efficiency comparison between our ViMD and other methods on Caltech-UCSD Birds 200 cub benchmark. The color and circle size indicate the model's accuracy and floating point operations,respectively.
• Figure 2: An overview of ViMD, which is mainly composed of an SRVM-Net (student), an HRVM-Net (teacher), and a multi-level Mamba knowledge distillation loss composed of $L_{HSD}$ and $L_{LD}$. In the training phase, HRVM-Net is firstly trained on HR images, and then SRVM-Net is trained on LR images under the supervision of the multi-level Mamba knowledge distillation loss with the help of HRVM-Net. In the test phase, only SRVM-Net is employed, and it can directly output the prediction results when LR images are given.
• Figure 3: Results of hyper-parameters analysis on the four datasets.