Ranking-aware Continual Learning for LiDAR Place Recognition

TL;DR

This work tackles catastrophic forgetting in continual LiDAR place recognition by introducing KDF, a framework built on ranking-aware knowledge distillation and a knowledge fusion module. RKD preserves the intrinsic ranking structure of place embeddings across sequential domains, while DKD aligns the overall feature distributions between old and new models; a triplet-based metric learning loss supports robust descriptor learning, and fusion consolidates knowledge from both models. The method demonstrates improved mean Recall@1 and reduced forgetting across four backbone networks and multiple datasets, including generalization to unseen KITTI sequences. Overall, KDF enables smoother knowledge transfer and stronger generalization in evolving environments, with potential to extend to visual and multimodal place recognition tasks.

Abstract

Place recognition plays a significant role in SLAM, robot navigation, and autonomous driving applications. Benefiting from deep learning, the performance of LiDAR place recognition (LPR) has been greatly improved. However, many existing learning-based LPR methods suffer from catastrophic forgetting, which severely harms the performance of LPR on previously trained places after training on a new environment. In this paper, we introduce a continual learning framework for LPR via Knowledge Distillation and Fusion (KDF) to alleviate forgetting. Inspired by the ranking process of place recognition retrieval, we present a ranking-aware knowledge distillation loss that encourages the network to preserve the high-level place recognition knowledge. We also introduce a knowledge fusion module to integrate the knowledge of old and new models for LiDAR place recognition. Our extensive experiments demonstrate that KDF can be applied to different networks to overcome catastrophic forgetting, surpassing the state-of-the-art methods in terms of mean Recall@1 and forgetting score.

Figures (4)

• Figure 1: The LiDAR place recognition (LPR) and continual LPR, and comparison of knowledge distillation strategies in different methods. (a) The LiDAR place recognition (LPR) get the best match of query sample by the ranking-based retrieval. (b) The LPR model learns from different environments and the catastrophic forgetting lead to performance degradation on old environment. (c) Comparison of knowledge distillation strategies between existing continual learning methods for LPR and our proposed method. Existing methods rely solely on response alignment between old and new models, such as angles (InCloud knights2022irosincloud) or distributions (CCL cui2023ralccl) between descriptors. In contrast, our method not only models the feature distributions of the old and new models but also captures the intrinsic relationship between features by incorporating ranking consistency.
• Figure 2: An overview of our proposed method KDF. We first initialize a new model from the old model and freeze the old model. Next, memory exemplars and new data are fed into both models for knowledge distillation and metric learning. After training, the acquired knowledge from both models is fused in preparation for future steps. Let $E_{old}$ and $E_{new}$ denote the embeddings from the old and new models, respectively. $\mathcal{L}_{PR}$ represents the place recognition loss, i.e., the triplet margin loss. $\mathcal{L}_{RKD}$ and $\mathcal{L}_{DKD}$ correspond to the ranking-based distillation loss and the distribution-based distillation loss, respectively. Yellow, blue, and red indicate steps $t-1$, $t$, and $t+1$, respectively.
• Figure 3: The ranking-based knowledge distillation loss $\mathcal{L}_{RKD}$ enforces the model to maintain more informative knowledge from the previous domain. Given two embedding sets $E^{t}$ and $E^{t+1}$ from the old and new models, we compute their internal ranking information $\mathcal{R}(q,i)^{t}$ and $\mathcal{R}(q,i)^{t+1}$ by similarities. Then, we minimize the difference between them by knowledge distillation.
• Figure 4: Detailed performance comparison in the $4$-$step$ protocol across PointNetVLAD, LoGG3D-Net, MinkLoc3D, and TransLoc3D with methods InCloud knights2022irosincloud, CCL cui2023ralccl, and MICL liu2024micl. The results are composed of Recall@1 matrices, where each row represents a continual learning method and each column represents a different LPR backbone. In each Recall@1 matrix, steps 1 to 4 record the Recall@1 results of the environments that have been visited. A darker color means higher Recall@1.