Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ghost-dil-NetVLAD: A Lightweight Neural Network for Visual Place Recognition

Qingyuan Gong, Yu Liu, Liqiang Zhang, Renhe Liu

TL;DR

The paper tackles efficient visual place recognition (VPR) by proposing Ghost-dil-NetVLAD, a lightweight end-to-end network that combines GhostCNN for feature extraction with a learnable NetVLAD pooling back-end. By incorporating dilated convolutions within Ghost modules, the approach captures richer spatial context while maintaining a small computational footprint, achieving state-of-the-art efficiency. Empirical results show substantial reductions in FLOPs (≈99.04%) and parameters (≈80.16%) compared with VGG16-NetVLAD, with comparable accuracy on public benchmarks and the authors’ TJU-Location dataset, especially when trained with Places-365 pretraining. The work also demonstrates that SE blocks, targeted pretraining, and moderate dilation further boost performance, though challenges remain in low-light scenarios, pointing to future work on illumination robustness and deployment on embedded systems.

Abstract

Visual place recognition (VPR) is a challenging task with the unbalance between enormous computational cost and high recognition performance. Thanks to the practical feature extraction ability of the lightweight convolution neural networks (CNNs) and the train-ability of the vector of locally aggregated descriptors (VLAD) layer, we propose a lightweight weakly supervised end-to-end neural network consisting of a front-ended perception model called GhostCNN and a learnable VLAD layer as a back-end. GhostCNN is based on Ghost modules that are lightweight CNN-based architectures. They can generate redundant feature maps using linear operations instead of the traditional convolution process, making a good trade-off between computation resources and recognition accuracy. To enhance our proposed lightweight model further, we add dilated convolutions to the Ghost module to get features containing more spatial semantic information, improving accuracy. Finally, rich experiments conducted on a commonly used public benchmark and our private dataset validate that the proposed neural network reduces the FLOPs and parameters of VGG16-NetVLAD by 99.04% and 80.16%, respectively. Besides, both models achieve similar accuracy.

Ghost-dil-NetVLAD: A Lightweight Neural Network for Visual Place Recognition

TL;DR

The paper tackles efficient visual place recognition (VPR) by proposing Ghost-dil-NetVLAD, a lightweight end-to-end network that combines GhostCNN for feature extraction with a learnable NetVLAD pooling back-end. By incorporating dilated convolutions within Ghost modules, the approach captures richer spatial context while maintaining a small computational footprint, achieving state-of-the-art efficiency. Empirical results show substantial reductions in FLOPs (≈99.04%) and parameters (≈80.16%) compared with VGG16-NetVLAD, with comparable accuracy on public benchmarks and the authors’ TJU-Location dataset, especially when trained with Places-365 pretraining. The work also demonstrates that SE blocks, targeted pretraining, and moderate dilation further boost performance, though challenges remain in low-light scenarios, pointing to future work on illumination robustness and deployment on embedded systems.

Abstract

Visual place recognition (VPR) is a challenging task with the unbalance between enormous computational cost and high recognition performance. Thanks to the practical feature extraction ability of the lightweight convolution neural networks (CNNs) and the train-ability of the vector of locally aggregated descriptors (VLAD) layer, we propose a lightweight weakly supervised end-to-end neural network consisting of a front-ended perception model called GhostCNN and a learnable VLAD layer as a back-end. GhostCNN is based on Ghost modules that are lightweight CNN-based architectures. They can generate redundant feature maps using linear operations instead of the traditional convolution process, making a good trade-off between computation resources and recognition accuracy. To enhance our proposed lightweight model further, we add dilated convolutions to the Ghost module to get features containing more spatial semantic information, improving accuracy. Finally, rich experiments conducted on a commonly used public benchmark and our private dataset validate that the proposed neural network reduces the FLOPs and parameters of VGG16-NetVLAD by 99.04% and 80.16%, respectively. Besides, both models achieve similar accuracy.
Paper Structure (21 sections, 7 equations, 5 figures, 5 tables)

This paper contains 21 sections, 7 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Lightweight VPR Model
  4. Deep Learned Representations for VPR
  5. Lightweight CNN
  6. Methodology
  7. GhostCNN
  8. Ghost Bottleneck
  9. Ghost Module
  10. NetVLAD layer
  11. Experiments
  12. Datasets
  13. Evaluation Metrics
  14. Implementation Details
  15. Results and Discussion
  16. ...and 6 more sections

Figures (5)

  • Figure 1: The proposed framework for place recognition. The feature extraction part is lightweight CNN architecture, which is a 5-stage CNN architecture. Through every stage, the feature map becomes a half one. The NetVLAD layer processes local descriptors to global descriptors.
  • Figure 2: Ghost bottleneck architecture.
  • Figure 3: Ghost module with dilated convolution filter. We add dilated convolutions to the first step of the Ghost module. Sub-figure (A) shows the $3 \times 3$ receptive field of 1-dilated convolution filter (i.e., ordinary convolution). (B) reveals the $7 \times 7$ receptive field of 2-dilated convolution filter. (C) indicates the $11 \times 11$ receptive field of 3-dilated convolution filter.
  • Figure 4: TJU-Location dataset examples. Each column shows perspective images taken at different times.
  • Figure 5: FLOPs and parameters of the Alex-NetVLAD, VGG16-NetVLAD, Patch-NetVLAD, MobileNetV3-NetVLAD, Ghost-NetVLAD and Ghost-dil-NetVLAD.