Deep Learning Based Named Entity Recognition Models for Recipes

TL;DR

This work addresses recipe text NER by building and leveraging three labeled data resources (Manually_Annotated, Augmented, and Machine_Annotated) derived from RecipeDB, augmented with three data-augmentation strategies and a SEFS sampling approach to ensure diversity. It systematically benchmarks a spectrum of models—from Stanford NER CRF baselines to encoder-based architectures (BERT, DistilBERT, RoBERTa variants) and spaCy/Flair frameworks—evaluating them with macro-$F1$, precision, and recall across dataset strata. The study finds that fine-tuned spaCy-transformer delivers the strongest performance (macro-$F1$ around $95.9\%$ to $96.04\%$ across datasets), while few-shot prompting on large language models yields poor results in this domain. These results have practical implications for scalable information extraction in culinary data, enabling more accurate ingredient recognition, flavor profiling, and nutrition-related analytics, and suggest directions for domain-adapted LLM fine-tuning and multilingual NER in recipes.

Abstract

Food touches our lives through various endeavors, including flavor, nourishment, health, and sustainability. Recipes are cultural capsules transmitted across generations via unstructured text. Automated protocols for recognizing named entities, the building blocks of recipe text, are of immense value for various applications ranging from information extraction to novel recipe generation. Named entity recognition is a technique for extracting information from unstructured or semi-structured data with known labels. Starting with manually-annotated data of 6,611 ingredient phrases, we created an augmented dataset of 26,445 phrases cumulatively. Simultaneously, we systematically cleaned and analyzed ingredient phrases from RecipeDB, the gold-standard recipe data repository, and annotated them using the Stanford NER. Based on the analysis, we sampled a subset of 88,526 phrases using a clustering-based approach while preserving the diversity to create the machine-annotated dataset. A thorough investigation of NER approaches on these three datasets involving statistical, fine-tuning of deep learning-based language models and few-shot prompting on large language models (LLMs) provides deep insights. We conclude that few-shot prompting on LLMs has abysmal performance, whereas the fine-tuned spaCy-transformer emerges as the best model with macro-F1 scores of 95.9%, 96.04%, and 95.71% for the manually-annotated, augmented, and machine-annotated datasets, respectively.

Figures (8)

• Figure 1: The pipeline implemented for fine-tuning supervised deep learning-based named entity recognition comprises three phases. To begin with, we built vocabularies for each of the three datasets. Further, we utilized these vocabularies to convert every word of an input sentence into corresponding token indexes that were subsequently converted to token embeddings via tokenization. Finally, Encoder-Only language models were employed to predict entity tags for the input token embeddings. The spaCy-transformer emerged as the best model with F1 scores of 95.9%, 96.04%, and 95.71% for Manually_Annotated, Augmented, and Machine_Annotated datasets, respectively.
• Figure 2: Illustration of Data Augmentation strategies to generate new samples. (a) LwTR: Labelwise Token Replacement: replace a token with a random token of the same label. (b) SR: Synonym Replacement: replace a token with its synonym from Wordnet. (c) SiS: Shuffle within segments: shuffle the tokens under their corresponding label within an ingredient phrase.
• Figure 3: Analysis of the percentage of ingredient phrases captured by various clusters. The distribution is extremely skewed, with a few clusters hoarding most ingredient phrases. Half of the ingredient phrases, for example, are captured by merely the eleven largest clusters.
• Figure 4: Model Comparison based on F1-scores and Loss. (a), (b) and (c) represent epoch-wise F1-score for Manually Annotated, Augmented, and Machine Annotated Datasets, respectively. Similarly, (d), (e) and (f) represent the epoch-wise Loss score for three datasets.
• Figure 5: Tag-wise learnability of named entities and their final results using the best-performing model--the spaCy-transformer. Figures (a), (b) and (c) depict these results for the Manually_Annotated, Augmented, and Machine_Augmented datasets, respectively.