Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero- and Few-Shot Named-Entity Recognition: Case Study and Dataset in the Crime Domain (CrimeNER)

Miguel Lopez-Duran, Julian Fierrez, Aythami Morales, Daniel DeAlcala, Gonzalo Mancera, Javier Irigoyen, Ruben Tolosana, Oscar Delgado, Francisco Jurado, Alvaro Ortigosa

TL;DR

CrimeNER, a case-study of Crime-related zero- and Few-Shot NER, and a general Crime-related Named-Entity Recognition database (CrimeNERdb) consisting of more than 1.5k annotated documents for the NER task extracted from public reports on terrorist attacks and the U.S. Department of Justice's press notes are presented.

Abstract

The extraction of critical information from crime-related documents is a crucial task for law enforcement agencies. Named-Entity Recognition (NER) can perform this task in extracting information about the crime, the criminal, or law enforcement agencies involved. However, there is a considerable lack of adequately annotated data on general real-world crime scenarios. To address this issue, we present CrimeNER, a case-study of Crime-related zero- and Few-Shot NER, and a general Crime-related Named-Entity Recognition database (CrimeNERdb) consisting of more than 1.5k annotated documents for the NER task extracted from public reports on terrorist attacks and the U.S. Department of Justice's press notes. We define 5 types of coarse crime entity and a total of 22 types of fine-grained entity. We address the quality of the case-study and the annotated data with experiments on Zero and Few-Shot settings with State-of-the-Art NER models as well as generalist and commonly used Large Language Models.

Zero- and Few-Shot Named-Entity Recognition: Case Study and Dataset in the Crime Domain (CrimeNER)

TL;DR

CrimeNER, a case-study of Crime-related zero- and Few-Shot NER, and a general Crime-related Named-Entity Recognition database (CrimeNERdb) consisting of more than 1.5k annotated documents for the NER task extracted from public reports on terrorist attacks and the U.S. Department of Justice's press notes are presented.

Abstract

The extraction of critical information from crime-related documents is a crucial task for law enforcement agencies. Named-Entity Recognition (NER) can perform this task in extracting information about the crime, the criminal, or law enforcement agencies involved. However, there is a considerable lack of adequately annotated data on general real-world crime scenarios. To address this issue, we present CrimeNER, a case-study of Crime-related zero- and Few-Shot NER, and a general Crime-related Named-Entity Recognition database (CrimeNERdb) consisting of more than 1.5k annotated documents for the NER task extracted from public reports on terrorist attacks and the U.S. Department of Justice's press notes. We define 5 types of coarse crime entity and a total of 22 types of fine-grained entity. We address the quality of the case-study and the annotated data with experiments on Zero and Few-Shot settings with State-of-the-Art NER models as well as generalist and commonly used Large Language Models.
Paper Structure (15 sections, 3 figures, 5 tables)

This paper contains 15 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Zero and Few-Shot NER
  4. NER on Low Resource Domains
  5. NLP for Security and Crime tasks
  6. CrimeNER database
  7. Data acquisition and preprocessing
  8. Annotation process
  9. CrimeNERdb statistics
  10. Experiments and Results
  11. Zero-Shot
  12. Few-Shot
  13. Conclusions and Future Work

Figures (3)

  • Figure 1: Example of an annotated document on Doccano with both coarse and fine-grained entity types and nested entities.
  • Figure 2: Fine-grained entity span distribution of CrimeNERdb. PER and ORG refer to Person and Organization respectively.
  • Figure 3: Zero and Few-Shot mean F1-scores for coarse entity types with GPT-4.1-Mini(Fig. \ref{['fig::zero and few shot results']}(a)) and Zero and Few-Shot mean F1-scores for fine-grained entity types with GPT-4o-Mini(Fig. \ref{['fig::zero and few shot results']}(b)).