Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Logic Rules as Explanations for Legal Case Retrieval

Zhongxiang Sun, Kepu Zhang, Weijie Yu, Haoyu Wang, Jun Xu

TL;DR

This paper proposes Neural-Symbolic enhanced Legal Case Retrieval (NS-LCR), a framework that explicitly conducts reasoning on the matching of legal cases through learning case-level and law-level logic rules and is equipped with built-in faithful explainability.

Abstract

In this paper, we address the issue of using logic rules to explain the results from legal case retrieval. The task is critical to legal case retrieval because the users (e.g., lawyers or judges) are highly specialized and require the system to provide logical, faithful, and interpretable explanations before making legal decisions. Recently, research efforts have been made to learn explainable legal case retrieval models. However, these methods usually select rationales (key sentences) from the legal cases as explanations, failing to provide faithful and logically correct explanations. In this paper, we propose Neural-Symbolic enhanced Legal Case Retrieval (NS-LCR), a framework that explicitly conducts reasoning on the matching of legal cases through learning case-level and law-level logic rules. The learned rules are then integrated into the retrieval process in a neuro-symbolic manner. Benefiting from the logic and interpretable nature of the logic rules, NS-LCR is equipped with built-in faithful explainability. We also show that NS-LCR is a model-agnostic framework that can be plugged in for multiple legal retrieval models. To showcase NS-LCR's superiority, we enhance existing benchmarks by adding manually annotated logic rules and introducing a novel explainability metric using Large Language Models (LLMs). Our comprehensive experiments reveal NS-LCR's effectiveness for ranking, alongside its proficiency in delivering reliable explanations for legal case retrieval.

Logic Rules as Explanations for Legal Case Retrieval

TL;DR

This paper proposes Neural-Symbolic enhanced Legal Case Retrieval (NS-LCR), a framework that explicitly conducts reasoning on the matching of legal cases through learning case-level and law-level logic rules and is equipped with built-in faithful explainability.

Abstract

In this paper, we address the issue of using logic rules to explain the results from legal case retrieval. The task is critical to legal case retrieval because the users (e.g., lawyers or judges) are highly specialized and require the system to provide logical, faithful, and interpretable explanations before making legal decisions. Recently, research efforts have been made to learn explainable legal case retrieval models. However, these methods usually select rationales (key sentences) from the legal cases as explanations, failing to provide faithful and logically correct explanations. In this paper, we propose Neural-Symbolic enhanced Legal Case Retrieval (NS-LCR), a framework that explicitly conducts reasoning on the matching of legal cases through learning case-level and law-level logic rules. The learned rules are then integrated into the retrieval process in a neuro-symbolic manner. Benefiting from the logic and interpretable nature of the logic rules, NS-LCR is equipped with built-in faithful explainability. We also show that NS-LCR is a model-agnostic framework that can be plugged in for multiple legal retrieval models. To showcase NS-LCR's superiority, we enhance existing benchmarks by adding manually annotated logic rules and introducing a novel explainability metric using Large Language Models (LLMs). Our comprehensive experiments reveal NS-LCR's effectiveness for ranking, alongside its proficiency in delivering reliable explanations for legal case retrieval.
Paper Structure (32 sections, 12 equations, 3 figures, 8 tables)

This paper contains 32 sections, 12 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Legal Case Retrieval
  4. Logic as Explanation
  5. Background and Preliminaries
  6. Task Formulation
  7. Presenting law articles as FOL
  8. Fuzzy Logic
  9. OUR APPROACH: NS-LCR
  10. General framework
  11. Law-level Module
  12. Law-level Rule Evaluation
  13. Law-level Explanation and Induction
  14. Case-level Module
  15. Case-level Rule Evaluation
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Explanations provided by different legal case retrieval models. Semantic models (e.g., Lawformer) only estimate the matching score. Existing explainable methods (e.g., IOT-Match) provide sentences as explanations. NS-LCR aims to explain matching results with case and law logic rules. Article 264 of PRC Criminal Law, with three key facts $P_{1}$, $P_{2}$, and $P_{3}$, applies to the target case.
  • Figure 2: The overall architecture of the proposed model NS-LCR.
  • Figure 3: Various base models' performance w/ and wo/ NS-LCR on LeCaRD is shown. The lines show the NDCG@30 scores and the bars show the improvement percentage due to NS-LCR.