BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning

TL;DR

This paper tackles the latency and cognitive load problem in retrieval-augmented generation when handling long contexts for multi-hop QA. It introduces BRIEF-Pro, a lightweight compressor trained on a synthetic long-context data pipeline built from short-context seeds, capable of user-controllable or automatic summary lengths. The approach yields substantial compression (around 32x) and improves QA accuracy across small, large, and proprietary readers, while reducing end-to-end latency and computational overhead. The work demonstrates practical impact for scalable RAG systems and outlines a data-centric path to long-context summarization with broad applicability, alongside discussions of limitations and future directions.

Abstract

As retrieval-augmented generation (RAG) tackles complex tasks, increasingly expanded contexts offer richer information, but at the cost of higher latency and increased cognitive load on the model. To mitigate this bottleneck, especially for intricate multi-hop questions, we introduce BRIEF-Pro. It is a universal, lightweight compressor that distills relevant evidence for a given query from retrieved documents into a concise summary for seamless integration into in-context RAG. Using seed data consisting of relatively short contexts (fewer than 1k words), BRIEF-Pro is trained to perform abstractive compression of extended contexts exceeding 10k words across a wide range of scenarios. Furthermore, BRIEF-Pro offers flexible user control over summary length by allowing users to specify the desired number of sentences. Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. With the 70B reader model, 32x compression by BRIEF-Pro improves QA performance by 4.67% on average over LongLLMLingua's 9x, while requiring only 23% of its computational overhead.

Figures (6)

• Figure 1: A comparison of the inference process with and without the lightweight BRIEF-Pro. Retrieved documents are compressed into a highly dense textual summary relevant to the query before being prepended, thereby reducing the cognitive load caused by the extended context on a range of larger models, including 8B, 70B, and proprietary models.
• Figure 2: An overview of the synthetic data pipeline for training BRIEF-Pro. Starting with a mixture of oracle and distractor documents for a given query, the pipeline: (a) expands each document by looking up an external knowledge corpus, (b) curates a compact summary by further reducing redundancy in the oracle documents, and (c) generates a user-controllable compression instruction by counting the number of sentences in the compact summary.
• Figure 3: The comparison of TFLOPs consumption using (a) Llama-3.1-8B-Instruct and (b) Llama-3.1-70B-Instruct as the reader model.
• Figure 4: The distribution of sentence counts in the generated summary over four test sets in various compression modes.
• Figure 5: The performance of compressors under different context length. We expand the scope of retrieved documents from the top 20 to the top 100 based on the validation set in the Musique dataset DBLP:journals/tacl/TrivediBKS22. For BRIEF-PRO, we follow the AUTO setting. For LongLLMLingua DBLP:conf/acl/JiangWL0L0Q24, we follow their 2,000-token compression constraint. The reported score is the average of the F1 and EM scores.