Fine Tuning Named Entity Extraction Models for the Fantasy Domain
Aravinth Sivaganeshan, Nisansa de Silva
TL;DR
The paper tackles the challenge of recognizing domain-specific entities in fantasy text by fine-tuning a high-capacity NER model (Trankit) on monster lore from Dungeons & Dragons. It introduces two data-collection and tagging strategies to create BIO-formatted training data, and compares fine-tuned Trankit against zero-shot Trankit and two Flair configurations, using both a text-lookup baseline and a gold-standard test set. The study finds that the best fine-tuned Trankit achieves an F1 of 87.86% on gold-standard data, with FRW-I data aiding performance and Setup 2 outperforming Setup 1; Flair models provide competitive recall and F1 as well. These results demonstrate the effectiveness of domain-specific fine-tuning for niche entity types and support downstream applications such as D&D encounter generation and lore-based information extraction.
Abstract
Named Entity Recognition (NER) is a sequence classification Natural Language Processing task where entities are identified in the text and classified into predefined categories. It acts as a foundation for most information extraction systems. Dungeons and Dragons (D&D) is an open-ended tabletop fantasy game with its own diverse lore. DnD entities are domain-specific and are thus unrecognizable by even the state-of-the-art off-the-shelf NER systems as the NER systems are trained on general data for pre-defined categories such as: person (PERS), location (LOC), organization (ORG), and miscellaneous (MISC). For meaningful extraction of information from fantasy text, the entities need to be classified into domain-specific entity categories as well as the models be fine-tuned on a domain-relevant corpus. This work uses available lore of monsters in the D&D domain to fine-tune Trankit, which is a prolific NER framework that uses a pre-trained model for NER. Upon this training, the system acquires the ability to extract monster names from relevant domain documents under a novel NER tag. This work compares the accuracy of the monster name identification against; the zero-shot Trankit model and two FLAIR models. The fine-tuned Trankit model achieves an 87.86% F1 score surpassing all the other considered models.