Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains

TL;DR

Domain$o1$s addresses the need for explainable reasoning in high-stakes domains by extending LLMs with two domain-specific variants and a scalable tree-search-based reasoning path exploration. The method combines supervised fine-tuning on CoT-style datasets (CoT-stock-2k, CoT-legal-2k) with Selective Tree Exploration and a novel PROOF-Score metric to evaluate explainability beyond accuracy. Empirical results on stock investment and legal reasoning QA show improved reasoning accuracy and richer, verifiable explanations compared to base models and conventional CoT approaches. The work offers practical impact for decision support in finance and law and provides open-source resources to enable further research and application.

Abstract

Large Language Models (LLMs) are widely applied to downstream domains. However, current LLMs for high-stakes domain tasks, such as financial investment and legal QA, typically generate brief answers without reasoning processes and explanations. This limits users' confidence in making decisions based on their responses. While original CoT shows promise, it lacks self-correction mechanisms during reasoning. This work introduces Domain$o1$s, which enhances LLMs' reasoning capabilities on domain tasks through supervised fine-tuning and tree search. We construct CoT-stock-2k and CoT-legal-2k datasets for fine-tuning models that activate domain-specific reasoning steps based on their judgment. Additionally, we propose Selective Tree Exploration to spontaneously explore solution spaces and sample optimal reasoning paths to improve performance. We also introduce PROOF-Score, a new metric for evaluating domain models' explainability, complementing traditional accuracy metrics with richer assessment dimensions. Extensive experiments on stock investment recommendation and legal reasoning QA tasks demonstrate Domaino1s's leading performance and explainability. Our code is available at https://github.com/Hyalinesky/Domaino1s.

Figures (15)

• Figure 1: Comparison of Domain$o1$s and other paradigms on a demonstrative example. Domain$o1$s expands reasoning paths and obtains optimal ones through tree search.
• Figure 2: Comparison between the base model Qwen-2.5-Instruct qwen2.5 and Domain$o1$s. The base model shows notable reasoning errors. In contrast, Domain$o1$s breaks problems into multiple reasoning steps and reaches well-supported conclusions through systematic analysis. Details in Appendix \ref{['Appendix_Answer_Demonstration']}.
• Figure 3: Solution expansion & sampling illustration. Best-of-N search generates N complete responses and selects the best one; Sentence-level Beam Search generates multiple candidates for each sentence and selects the best one; Similarly, Stage-level Beam Search generates multiple candidates for each reasoning step and selects the best one. In contrast, our Selective Tree Exploration dynamically expands each reasoning step node, explores multiple reasoning steps as candidates only when necessary, and selects the best option at each step. Our method balances search performance and computational time overhead.
• Figure 4: The role of solution expansion & sampling. Intermediate steps in single inference (without sample) may contain errors, while solution expansion & sampling can find better reasoning paths.
• Figure 5: Models responses and accuracy on legal reasoning QA tasks after removing key conditions.