Enhancing Retrieval Systems with Inference-Time Logical Reasoning

TL;DR

The paper tackles the limitation of cosine-based retrieval in handling complex queries by introducing an inference-time logical reasoning framework (ITLR). ITLR converts natural language queries into a logical form via an LLM, embeds individual terms, and computes a final document score by composing per-term similarities through a logic-guided operator tree, e.g., $s_i = OP_{AND}(OP_{OR}(s_{1i}, s_{2i}), OP_{NOT}(s_{3i}))$. Through synthetic and BEIR real-data experiments across multiple embedding models, ITLR consistently outperforms traditional retrieval baselines, with especially large gains for queries featuring negations and conjunctions. The approach maintains efficiency by enabling parallel term embeddings and a simple fuzzy evaluation of logical expressions, offering a practical enhancement for retrieval systems dealing with compositional queries.

Abstract

Traditional retrieval methods rely on transforming user queries into vector representations and retrieving documents based on cosine similarity within an embedding space. While efficient and scalable, this approach often fails to handle complex queries involving logical constructs such as negations, conjunctions, and disjunctions. In this paper, we propose a novel inference-time logical reasoning framework that explicitly incorporates logical reasoning into the retrieval process. Our method extracts logical reasoning structures from natural language queries and then composes the individual cosine similarity scores to formulate the final document scores. This approach enables the retrieval process to handle complex logical reasoning without compromising computational efficiency. Our results on both synthetic and real-world benchmarks demonstrate that the proposed method consistently outperforms traditional retrieval methods across different models and datasets, significantly improving retrieval performance for complex queries.

Figures (4)

• Figure 1: Given a query "What are the benefits of vitamin D, focusing on benefits other than bone health?", we first convert the query into the logical expression "Vitamin D Benefits AND NOT Bone Health". We then calculate the cosine similarity scores for each term (top row) and combine these scores to generate the final results.
• Figure 2: Example parse tree (left) and corresponding graph of operations (right).
• Figure 3: Performance as the number of terms scales. Baseline dense retrieval and logical retrieval were evaluated on queries connected by AND AND and OR OR clauses, with increasing number of clauses.
• Figure 4: Expanded version of Fig. \ref{['fig:and_or']} including calibration.