SPLADE-v3: New baselines for SPLADE
Carlos Lassance, Hervé Déjean, Thibault Formal, Stéphane Clinchant
TL;DR
SPLADE-v3 introduces training-structure enhancements to SPLADE that yield strong first-stage retrieval baselines. It leverages multiple negatives per batch, ensemble distillation scores, and dual distillation losses (KL-Div and MarginMSE), with targeted fine-tuning starting from SPLADE++SelfDistil. Evaluations via a meta-analysis across 44 query sets show significant gains over BM25 and SPLADE++, and competitive performance with cross-encoder re-rankers on MS MARCO and BEIR. The work also explores efficiency-focused variants (DistilBERT, Lexical, Doc) to balance retrieval effectiveness and deployment cost, offering robust, scalable baselines for information retrieval systems.
Abstract
A companion to the release of the latest version of the SPLADE library. We describe changes to the training structure and present our latest series of models -- SPLADE-v3. We compare this new version to BM25, SPLADE++, as well as re-rankers, and showcase its effectiveness via a meta-analysis over more than 40 query sets. SPLADE-v3 further pushes the limit of SPLADE models: it is statistically significantly more effective than both BM25 and SPLADE++, while comparing well to cross-encoder re-rankers. Specifically, it gets more than 40 MRR@10 on the MS MARCO dev set, and improves by 2% the out-of-domain results on the BEIR benchmark.