Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum

Zhichao Xu, Shengyao Zhuang, Crystina Zhang, Xueguang Ma, Yijun Tian, Maitrey Mehta, Jimmy Lin, Vivek Srikumar

TL;DR

LACONIC advances sparse retrieval by leveraging Llama3 backbones with a two-phase training curriculum to achieve dense-level effectiveness on CPU-friendly inverted indices. The approach combines bidirectional adaptation, sparse vocabulary representations, and an InfoNCE objective with FLOPs regularization to train efficient sparse encoders. Empirical results show LACONIC-8B achieving 60.2 nDCG@10 on MTEB-R, while using substantially less index memory than dense models, highlighting practical deployment on commodity hardware. The work provides open-source models and data, demonstrating a scalable path for high-precision web-scale retrieval without heavy GPU requirements.

Abstract

While dense retrieval models have become the standard for state-of-the-art information retrieval, their deployment is often constrained by high memory requirements and reliance on GPU accelerators for vector similarity search. Learned sparse retrieval offers a compelling alternative by enabling efficient search via inverted indices, yet it has historically received less attention than dense approaches. In this report, we introduce LACONIC, a family of learned sparse retrievers based on the Llama-3 architecture (1B, 3B, and 8B). We propose a streamlined two-phase training curriculum consisting of (1) weakly supervised pre-finetuning to adapt causal LLMs for bidirectional contextualization and (2) high-signal finetuning using curated hard negatives. Our results demonstrate that LACONIC effectively bridges the performance gap with dense models: the 8B variant achieves a state-of-the-art 60.2 nDCG on the MTEB Retrieval benchmark, ranking 15th on the leaderboard as of January 1, 2026, while utilizing 71\% less index memory than an equivalent dense model. By delivering high retrieval effectiveness on commodity CPU hardware with a fraction of the compute budget required by competing models, LACONIC provides a scalable and efficient solution for real-world search applications.

LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum

TL;DR

LACONIC advances sparse retrieval by leveraging Llama3 backbones with a two-phase training curriculum to achieve dense-level effectiveness on CPU-friendly inverted indices. The approach combines bidirectional adaptation, sparse vocabulary representations, and an InfoNCE objective with FLOPs regularization to train efficient sparse encoders. Empirical results show LACONIC-8B achieving 60.2 nDCG@10 on MTEB-R, while using substantially less index memory than dense models, highlighting practical deployment on commodity hardware. The work provides open-source models and data, demonstrating a scalable path for high-precision web-scale retrieval without heavy GPU requirements.

Abstract

While dense retrieval models have become the standard for state-of-the-art information retrieval, their deployment is often constrained by high memory requirements and reliance on GPU accelerators for vector similarity search. Learned sparse retrieval offers a compelling alternative by enabling efficient search via inverted indices, yet it has historically received less attention than dense approaches. In this report, we introduce LACONIC, a family of learned sparse retrievers based on the Llama-3 architecture (1B, 3B, and 8B). We propose a streamlined two-phase training curriculum consisting of (1) weakly supervised pre-finetuning to adapt causal LLMs for bidirectional contextualization and (2) high-signal finetuning using curated hard negatives. Our results demonstrate that LACONIC effectively bridges the performance gap with dense models: the 8B variant achieves a state-of-the-art 60.2 nDCG on the MTEB Retrieval benchmark, ranking 15th on the leaderboard as of January 1, 2026, while utilizing 71\% less index memory than an equivalent dense model. By delivering high retrieval effectiveness on commodity CPU hardware with a fraction of the compute budget required by competing models, LACONIC provides a scalable and efficient solution for real-world search applications.
Paper Structure (21 sections, 3 equations, 1 figure, 2 tables)

This paper contains 21 sections, 3 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Model Architecture
  4. Training Objective
  5. Pre-finetuning
  6. Dataset
  7. Training
  8. Finetuning
  9. Dataset
  10. Training
  11. Experiments
  12. Experimental Setup
  13. Implementation Details
  14. Baselines
  15. Indexing
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Efficiency comparison. Left plot shows the index search latency on MSMARCO dataset, measured by queries per second, versus retrieval performance on MTEB-R benchmark. Right plot shows memory requirement to load retrieval index. Notice that LACONIC improves the performance-latency frontier compared to baselines without requiring accelerators for efficient index search. We reproduce SPLADE-v3's result using Seismic.