DuMapper: Towards Automatic Verification of Large-Scale POIs with Street Views at Baidu Maps

TL;DR

This paper proposes DuMapper II, a highly efficient framework to accelerate POI verification by means of deep multimodal embedding and approximate nearest neighbor (ANN) search, which takes the signboard image and the coordinates of a real-world place as input to generate a low-dimensional vector, which can be leveraged by ANN algorithms to conduct a more accurate search through billions of archived POIs in the database within milliseconds.

Abstract

With the increased popularity of mobile devices, Web mapping services have become an indispensable tool in our daily lives. To provide user-satisfied services, such as location searches, the point of interest (POI) database is the fundamental infrastructure, as it archives multimodal information on billions of geographic locations closely related to people's lives, such as a shop or a bank. Therefore, verifying the correctness of a large-scale POI database is vital. To achieve this goal, many industrial companies adopt volunteered geographic information (VGI) platforms that enable thousands of crowdworkers and expert mappers to verify POIs seamlessly; but to do so, they have to spend millions of dollars every year. To save the tremendous labor costs, we devised DuMapper, an automatic system for large-scale POI verification with the multimodal street-view data at Baidu Maps. DuMapper takes the signboard image and the coordinates of a real-world place as input to generate a low-dimensional vector, which can be leveraged by ANN algorithms to conduct a more accurate search through billions of archived POIs in the database for verification within milliseconds. It can significantly increase the throughput of POI verification by $50$ times. DuMapper has already been deployed in production since \DuMPOnline, which dramatically improves the productivity and efficiency of POI verification at Baidu Maps. As of December 31, 2021, it has enacted over $405$ million iterations of POI verification within a 3.5-year period, representing an approximate workload of $800$ high-performance expert mappers.

Figures (4)

• Figure 1: A screenshot of the result of a POI search at Baidu Maps, a well-known Web mapping service in China. In this case, the user-desired POI has been verified by the street-view data where the signboard image, the name, and the coordinates of that POI are included.
• Figure 2: DuMapper is designed to be an intelligent agent that can continuously verify POIs with the multimodal street-view data at the VGI platform, in place of thousands of expert mappers.
• Figure 3: DuMapper I is the original automatic framework which imitates the process of POI verification conducted by expert mappers at Baidu Maps. It adopts a three-stage pipeline, i.e., geo-spatial index (GSI), optical character recognition (OCR), and candidate POI rank (CPR), to deal with the coordinates, as well as the signboards in street-view data for automatic POI verification.
• Figure 4: DuMapper II revolutionizes the way to attain large-scale POI verification by means of two advanced modules: deep multimodal embedding (DME) of a POI and approximate nearest neighbor (ANN) search through the large-scale POI database. This novel framework can highly accelerate the speed of automatic POI verification.