Duluth at SemEval-2025 Task 7: TF-IDF with Optimized Vector Dimensions for Multilingual Fact-Checked Claim Retrieval

TL;DR

The paper tackles multilingual fact-checked claim retrieval under SemEval-2025 Task 7 by implementing a TF-IDF based retrieval system with careful tuning of vector dimensions and tokenization. The best configuration uses word-level tokenization with a 15,000-feature vocabulary, achieving dev/test average S@10 scores of 0.78 and 0.69 across ten languages, though it lags behind the top system (0.96). The findings highlight that while modern neural models are dominant, a well-optimized traditional method can be a strong baseline, particularly in limited compute settings. The work includes a detailed data cleaning pipeline, a full retrieval workflow, and analysis showing language-resource effects, with future work proposing hybrid approaches to improve cross-lingual performance and robustness across languages.

Abstract

This paper presents the Duluth approach to the SemEval-2025 Task 7 on Multilingual and Crosslingual Fact-Checked Claim Retrieval. We implemented a TF-IDF-based retrieval system with experimentation on vector dimensions and tokenization strategies. Our best-performing configuration used word-level tokenization with a vocabulary size of 15,000 features, achieving an average success@10 score of 0.78 on the development set and 0.69 on the test set across ten languages. Our system showed stronger performance on higher-resource languages but still lagged significantly behind the top-ranked system, which achieved 0.96 average success@10. Our findings suggest that though advanced neural architectures are increasingly dominant in multilingual retrieval tasks, properly optimized traditional methods like TF-IDF remain competitive baselines, especially in limited compute resource scenarios.