RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrieval

Abstract

Scientific reasoning requires not only long-chain reasoning processes, but also knowledge of domain-specific terminologies and adaptation to updated findings. To deal with these challenges for scientific reasoning, we introduce RAISE, a step-by-step retrieval-augmented framework which retrieves logically relevant documents from in-the-wild corpus. RAISE is divided into three steps: problem decomposition, logical query generation, and logical retrieval. We observe that RAISE consistently outperforms other baselines on scientific reasoning benchmarks. We analyze that unlike other baselines, RAISE retrieves documents that are not only similar in terms of the domain knowledge, but also documents logically more relevant.

Figures (16)

• Figure 1: Overview of RAISE. RAISE is divided into three steps: (1) Problem Decomposition, (2) Logical Query Generation, and (3) Logical Retrieval.
• Figure 2: Performance comparison between RAISE-Direct and RAISE across datasets.
• Figure 3: Examples where RAISE-retrieved documents provide logically relevant information for scientific reasoning compared to baseline RAG retrieval.
• Figure 4: Examples comparing query generation methods (Step-Back+RAG, HyDE, and RAISE) for the same subquestion. Both Step-Back+RAG and HyDE are methods that reformulate the original query to retrieve more relevant documents. These methods are included as baselines in the main comparison table.
• Figure 5: Logical Relevancy of Retrieved Documents. Unlike other baselines, RAISE has higher ratio of documents that are logically relevant and lower ratio of documents that are irrelevant or superficially relevant.