Legal$Δ$: Enhancing Legal Reasoning in LLMs via Reinforcement Learning with Chain-of-Thought Guided Information Gain

TL;DR

The paper addresses the problem that legal LLMs often produce direct, non-transparent answers lacking rigorous stepwise reasoning. It proposes LegalΔ, a reinforcement learning framework that injects chain-of-thought guided information gain into a GRPO-based RLVR pipeline, featuring a two-stage distill-and-refine process and a novel information-gain reward. The reward leverages the logits-to-Q-value equivalence to quantify reasoning quality and combines PMI between reasoning steps and the final answer with a global confidence term, encouraging robust, legally aligned reasoning. Experiments across five legal reasoning tasks and DiscLaw show about a 10% average improvement over strong baselines and good out-of-domain generalization, with ablations validating the importance of GRPO, SFT, and the information-gain mechanism for trustworthy legal judgments.

Abstract

Legal Artificial Intelligence (LegalAI) has achieved notable advances in automating judicial decision-making with the support of Large Language Models (LLMs). However, existing legal LLMs still struggle to generate reliable and interpretable reasoning processes. They often default to fast-thinking behavior by producing direct answers without explicit multi-step reasoning, limiting their effectiveness in complex legal scenarios that demand rigorous justification. To address this challenge, we propose Legal$Δ$, a reinforcement learning framework designed to enhance legal reasoning through chain-of-thought guided information gain. During training, Legal$Δ$ employs a dual-mode input setup-comprising direct answer and reasoning-augmented modes-and maximizes the information gain between them. This encourages the model to acquire meaningful reasoning patterns rather than generating superficial or redundant explanations. Legal$Δ$ follows a two-stage approach: (1) distilling latent reasoning capabilities from a powerful Large Reasoning Model (LRM), DeepSeek-R1, and (2) refining reasoning quality via differential comparisons, combined with a multidimensional reward mechanism that assesses both structural coherence and legal-domain specificity. Experimental results on multiple legal reasoning tasks demonstrate that Legal$Δ$ outperforms strong baselines in both accuracy and interpretability. It consistently produces more robust and trustworthy legal judgments without relying on labeled preference data. All code and data will be released at https://github.com/NEUIR/LegalDelta.

Figures (2)

• Figure 1: Illustration of Legal$\Delta$.
• Figure 2: Performance of Information-Gain Reward Modeling.