Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery

TL;DR

CNER-UAV is presented, a fine-grained dataset specifically designed for the task of address resolution in UAV delivery systems, enabling comprehensive training and evaluation of NER models.

Abstract

We present CNER-UAV, a fine-grained \textbf{C}hinese \textbf{N}ame \textbf{E}ntity \textbf{R}ecognition dataset specifically designed for the task of address resolution in \textbf{U}nmanned \textbf{A}erial \textbf{V}ehicle delivery systems. The dataset encompasses a diverse range of five categories, enabling comprehensive training and evaluation of NER models. To construct this dataset, we sourced the data from a real-world UAV delivery system and conducted a rigorous data cleaning and desensitization process to ensure privacy and data integrity. The resulting dataset, consisting of around 12,000 annotated samples, underwent human experts and \textbf{L}arge \textbf{L}anguage \textbf{M}odel annotation. We evaluated classical NER models on our dataset and provided in-depth analysis. The dataset and models are publicly available at \url{https://github.com/zhhvvv/CNER-UAV}.

Figures (2)

• Figure 1: The pipeline of dataset construction.
• Figure 2: Bad Cases of CNER-UAV-G and CNER-UAV-L datasets, where labels from CNER-UAV-G dataset are shown as G-tag, CNER-UAV-L as L-tag,CNER-UAV-H as H-tag