Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendation

Baharul Islam, Nasim Ahmad, Ferdous Ahmed Barbhuiya, Kuntal Dey

TL;DR

This work tackles cross-lingual subject classification for English and German scholarly articles by leveraging fixed embeddings for subjects and articles and training a lightweight transformation block with a margin-based objective to align them. The method uses a dimension-as-token Burst Attention layer on top of Sentence Transformer embeddings, with fixed subject vectors and a small 16-dimensional projection to model interdimensional relations. It achieves competitive recall under resource constraints, including an average Recall@k of 0.3224 on the official test and strong qualitative performance on common labels, while highlighting challenges with synonym overlap and niche topics. The approach demonstrates that parameter-efficient cross-lingual retrieval can be effective for large taxonomies like GND, offering practical gains for digital libraries under limited compute.

Abstract

We present our system submission for SemEval 2025 Task 5, which focuses on cross-lingual subject classification in the English and German academic domains. Our approach leverages bilingual data during training, employing negative sampling and a margin-based retrieval objective. We demonstrate that a dimension-as-token self-attention mechanism designed with significantly reduced internal dimensions can effectively encode sentence embeddings for subject retrieval. In quantitative evaluation, our system achieved an average recall rate of 32.24% in the general quantitative setting (all subjects), 43.16% and 31.53% of the general qualitative evaluation methods with minimal GPU usage, highlighting their competitive performance. Our results demonstrate that our approach is effective in capturing relevant subject information under resource constraints, although there is still room for improvement.

NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendation

TL;DR

This work tackles cross-lingual subject classification for English and German scholarly articles by leveraging fixed embeddings for subjects and articles and training a lightweight transformation block with a margin-based objective to align them. The method uses a dimension-as-token Burst Attention layer on top of Sentence Transformer embeddings, with fixed subject vectors and a small 16-dimensional projection to model interdimensional relations. It achieves competitive recall under resource constraints, including an average Recall@k of 0.3224 on the official test and strong qualitative performance on common labels, while highlighting challenges with synonym overlap and niche topics. The approach demonstrates that parameter-efficient cross-lingual retrieval can be effective for large taxonomies like GND, offering practical gains for digital libraries under limited compute.

Abstract

We present our system submission for SemEval 2025 Task 5, which focuses on cross-lingual subject classification in the English and German academic domains. Our approach leverages bilingual data during training, employing negative sampling and a margin-based retrieval objective. We demonstrate that a dimension-as-token self-attention mechanism designed with significantly reduced internal dimensions can effectively encode sentence embeddings for subject retrieval. In quantitative evaluation, our system achieved an average recall rate of 32.24% in the general quantitative setting (all subjects), 43.16% and 31.53% of the general qualitative evaluation methods with minimal GPU usage, highlighting their competitive performance. Our results demonstrate that our approach is effective in capturing relevant subject information under resource constraints, although there is still room for improvement.
Paper Structure (21 sections, 11 equations, 1 figure, 3 tables)

This paper contains 21 sections, 11 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Task Description
  3. Dataset Description
  4. Proposed System
  5. Training procedure.
  6. Inference procedure.
  7. Design motivation.
  8. Data Preprocessing
  9. Subject corpus
  10. Article corpus
  11. Embedding models
  12. Dimension-as-Token Projection and Reshaping
  13. Feed-Forward MLP
  14. Training and Inference
  15. Inference
  16. ...and 6 more sections

Figures (1)

  • Figure 1: System Architecture. The model processes subject and article embeddings through a transformation module (Token Projection & Reshaping + MLP) and optimizes via margin-based loss