Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Framework

Ibrahim Bouabdallaoui, Fatima Guerouate, Samya Bouhaddour, Chaimae Saadi, Mohammed Sbihi

TL;DR

FewTopNER tackles the challenge of few-shot, cross-lingual named entity recognition by integrating topic modeling into a unified multilingual framework. It combines a shared XLM-RoBERTa-based encoder with a prototype-based NER branch and a topic modeling branch, linked by a Cross-Task Bridge and supported by language-specific calibration and contrastive alignment. The approach yields consistent gains in NER F1 and topic coherence (NPMI), with strong cross-lingual transfer performance and robust multilingual efficiency. Ablation studies confirm the critical roles of the shared encoder and cross-task integration, demonstrating that joint consideration of topic context improves both entity disambiguation and thematic representations. Overall, FewTopNER sets a new benchmark for few-shot multilingual NER and offers practical potential for low-resource applications in diverse languages.

Abstract

We introduce FewTopNER, a novel framework that integrates few-shot named entity recognition (NER) with topic-aware contextual modeling to address the challenges of cross-lingual and low-resource scenarios. FewTopNER leverages a shared multilingual encoder based on XLM-RoBERTa, augmented with language-specific calibration mechanisms, to generate robust contextual embeddings. The architecture comprises a prototype-based entity recognition branch, employing BiLSTM and Conditional Random Fields for sequence labeling, and a topic modeling branch that extracts document-level semantic features through hybrid probabilistic and neural methods. A cross-task bridge facilitates dynamic bidirectional attention and feature fusion between entity and topic representations, thereby enhancing entity disambiguation by incorporating global semantic context. Empirical evaluations on multilingual benchmarks across English, French, Spanish, German, and Italian demonstrate that FewTopNER significantly outperforms existing state-of-the-art few-shot NER models. In particular, the framework achieves improvements of 2.5-4.0 percentage points in F1 score and exhibits enhanced topic coherence, as measured by normalized pointwise mutual information. Ablation studies further confirm the critical contributions of the shared encoder and cross-task integration mechanisms to the overall performance. These results underscore the efficacy of incorporating topic-aware context into few-shot NER and highlight the potential of FewTopNER for robust cross-lingual applications in low-resource settings.

FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Framework

TL;DR

FewTopNER tackles the challenge of few-shot, cross-lingual named entity recognition by integrating topic modeling into a unified multilingual framework. It combines a shared XLM-RoBERTa-based encoder with a prototype-based NER branch and a topic modeling branch, linked by a Cross-Task Bridge and supported by language-specific calibration and contrastive alignment. The approach yields consistent gains in NER F1 and topic coherence (NPMI), with strong cross-lingual transfer performance and robust multilingual efficiency. Ablation studies confirm the critical roles of the shared encoder and cross-task integration, demonstrating that joint consideration of topic context improves both entity disambiguation and thematic representations. Overall, FewTopNER sets a new benchmark for few-shot multilingual NER and offers practical potential for low-resource applications in diverse languages.

Abstract

We introduce FewTopNER, a novel framework that integrates few-shot named entity recognition (NER) with topic-aware contextual modeling to address the challenges of cross-lingual and low-resource scenarios. FewTopNER leverages a shared multilingual encoder based on XLM-RoBERTa, augmented with language-specific calibration mechanisms, to generate robust contextual embeddings. The architecture comprises a prototype-based entity recognition branch, employing BiLSTM and Conditional Random Fields for sequence labeling, and a topic modeling branch that extracts document-level semantic features through hybrid probabilistic and neural methods. A cross-task bridge facilitates dynamic bidirectional attention and feature fusion between entity and topic representations, thereby enhancing entity disambiguation by incorporating global semantic context. Empirical evaluations on multilingual benchmarks across English, French, Spanish, German, and Italian demonstrate that FewTopNER significantly outperforms existing state-of-the-art few-shot NER models. In particular, the framework achieves improvements of 2.5-4.0 percentage points in F1 score and exhibits enhanced topic coherence, as measured by normalized pointwise mutual information. Ablation studies further confirm the critical contributions of the shared encoder and cross-task integration mechanisms to the overall performance. These results underscore the efficacy of incorporating topic-aware context into few-shot NER and highlight the potential of FewTopNER for robust cross-lingual applications in low-resource settings.

Paper Structure

This paper contains 68 sections, 62 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep Neural Approaches.
  4. Transfer Learning and Domain Adaptation in NER.
  5. Few-Shot Learning for NER.
  6. Meta-Learning Approaches in NER.
  7. Prototypical Networks and Model Fusion.
  8. Our Positioning.
  9. Methodology
  10. Model Architecture Overview
  11. Shared Encoder
  12. Initial Contextual Embeddings
  13. Language-Specific Calibration
  14. Contrastive Learning for Alignment
  15. Entity Recognition Branch
  16. ...and 53 more sections

Figures (2)

  • Figure 1: Entity-Type Specific Performance Breakdown for Multilingual Datasets.
  • Figure 2: Entity Recognition Accuracy Across Configurations.