TartuNLP at SemEval-2025 Task 5: Subject Tagging as Two-Stage Information Retrieval

TL;DR

This work tackles automatically tagging documents with relevant subjects from the GND taxonomy for SemEval-2025 Task 5 by framing it as a two-stage information retrieval problem. A bi-encoder retrieves a coarse set of candidate subject definitions, which a cross-encoder then re-ranks to produce fine-grained, similarity-based relevance scores. Results show that the cross-encoder re-ranking nearly doubles recall compared to using only the bi-encoder, though at a higher computational cost, and qualitative analysis highlights challenges when tags share similar names or lack explicit relational information. The approach uses multilingual sentence embeddings, ANN indexing, and a fine-tuned cross-encoder, with code and models released for reproducibility, offering a practical trade-off between retrieval quality and scalability for large subject taxonomies.

Abstract

We present our submission to the Task 5 of SemEval-2025 that aims to aid librarians in assigning subject tags to the library records by producing a list of likely relevant tags for a given document. We frame the task as an information retrieval problem, where the document content is used to retrieve subject tags from a large subject taxonomy. We leverage two types of encoder models to build a two-stage information retrieval system -- a bi-encoder for coarse-grained candidate extraction at the first stage, and a cross-encoder for fine-grained re-ranking at the second stage. This approach proved effective, demonstrating significant improvements in recall compared to single-stage methods and showing competitive results according to qualitative evaluation.