NocPlace: Nocturnal Visual Place Recognition via Generative and Inherited Knowledge Transfer

TL;DR

NocPlace leverages generative and inherited knowledge transfer to embed resilience against dazzling lights and extreme darkness in the global descriptor and improves the performance of Eigenplaces by 7.6% on Tokyo 24/7 Night and 16.8% on SVOX Night.

Abstract

Visual Place Recognition (VPR) is crucial in computer vision, aiming to retrieve database images similar to a query image from an extensive collection of known images. However, like many vision tasks, VPR always degrades at night due to the scarcity of nighttime images. Moreover, VPR needs to address the cross-domain problem of night-to-day rather than just the issue of a single nighttime domain. In response to these issues, we present NocPlace, which leverages generative and inherited knowledge transfer to embed resilience against dazzling lights and extreme darkness in the global descriptor. First, we establish a day-night urban scene dataset called NightCities, capturing diverse lighting variations and dark scenarios across 60 cities globally. Then, an image generation network is trained on this dataset and processes a large-scale VPR dataset, obtaining its nighttime version. Finally, VPR models are fine-tuned using descriptors inherited from themselves and night-style images, which builds explicit cross-domain contrastive relationships. Comprehensive experiments on various datasets demonstrate our contributions and the superiority of NocPlace. Without adding any real-time computing resources, NocPlace improves the performance of Eigenplaces by 7.6% on Tokyo 24/7 Night and 16.8% on SVOX Night.

Figures (9)

• Figure 1: A demo for NocPlace. The center of this figure presents a map of San Francisco. 2.8 million daytime database images and 466 nighttime query images from SF-XL cosplace are projected onto this map via GPS tags, corresponding to the dense cyan-blue and sparse red dots, respectively. Despite the dazzling lights and extreme darkness, our method achieves higher recall@N than previous methods.
• Figure 2: Potential Datasets for Day-to-Night Translation. From left to right, they represent (a) LOL lol_dataset, (b) RobotCar robotcar, (c) day2night day2night, and (d) Tokyo 24/7 247_dataset.
• Figure 3: Examples of the NightCities dataset. Only nighttime images are displayed here, and daytime images are shown in the supplementary material. The images are arranged from left to right and top to bottom, representing the following cities in order: Tokyo, Hong Kong, San Francisco, Paris, Milan, Sydney, Munich, Zurich, and Vancouver.
• Figure 4: Schematic Diagrams of NocPlace. In the training phase, we transfer the night knowledge into the training data and use the existing label information. We inherit the descriptors extracted from the corresponding data by the pre-trained daytime VPR model and use them to supervise the NocPlace model. During the test phase, NocPlace extracts features from the query images and queries them in the descriptor database extracted by the daytime VPR model.
• Figure 5: Qualitative results on the SF-XL Night dataset. Each column represents a query (first and fourth rows) and the first predicted result from the database. We can see that NocPlace handles extreme lighting variations better than previous methods.