Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction

TL;DR

The paper introduces Legal Fact Prediction (LFP) to derive legally binding facts from trial evidence, enabling legal judgment prediction (LJP) before facts are formally established. It presents LFPBench, the first benchmark for evaluating LFP and LFP-empowered LJP, built from 657 Chinese civil cases with claims, evidence, facts, and judgments. Through extensive experiments with multiple LLMs and three LJP pipelines, the study shows that state-of-the-art models struggle with LFP but that incorporating predicted facts can markedly close the gap between evidence-based and fact-based LJP (about a 38.5% average reduction in the performance gap). It also uncovers biases related to evidence presentation order and quantity, and highlights challenges in reasoning over conflicting or non-written evidence, suggesting future work on multimodal reasoning and larger, more diverse datasets. Overall, LFP represents a practical step toward more usable and transparent LJP systems in real-world legal practice.

Abstract

Legal judgment prediction (LJP), which enables litigants and their lawyers to forecast judgment outcomes and refine litigation strategies, has emerged as a crucial legal NLP task. Existing studies typically utilize legal facts, i.e., facts that have been established by evidence and determined by the judge, to predict the judgment. However, legal facts are often difficult to obtain in the early stages of litigation, significantly limiting the practical applicability of fact-based LJP. To address this limitation, we propose a novel legal NLP task: legal fact prediction (LFP), which takes the evidence submitted by litigants for trial as input to predict legal facts, thereby empowering fact-based LJP technologies to make predictions in the absence of ground-truth legal facts. We also propose the first benchmark dataset, LFPBench, for evaluating the LFP task. Our extensive experiments on LFPBench demonstrate the effectiveness of LFP-empowered LJP and highlight promising research directions for LFP.

Figures (6)

• Figure 1: Connection between the legal fact prediction (LFP) and legal judgment prediction (LJP) tasks and comparison of three pipelines of LJP: fact-based LJP, evidence-based LJP, and LFP-empowered LJP. Most existing studies focus on fact-based LJP, while evidence-based LJP and LFP-empowered LJP remain unexplored.
• Figure 2: A trial primarily addresses two tasks: determining legal facts and applying the law.
• Figure 3: A data sample from the LFPBench dataset featuring a house lease case. Both the plaintiff and the defendant submitted evidence to assist the judge in determining the legal facts. However, Evidence 4, Evidence 6, and Evidence 7 present conflicting information regarding the defendant's actual move-out date. Ultimately, according to Evidence 7, the judge determined that the defendant had not moved out before July 7 and had defaulted on the rent for June.
• Figure 4: Distribution of case types in the LFPBench dataset.
• Figure 5: The correlation between the LFP similarity and the LJP accuaracy. We leverage the DP-Prompt method utpala2023locally to generate rewritten legal facts with varying LFP similarities.