Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate

Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori

TL;DR

The paper evaluates ChatGPT for extracting Principles of Law from Italian judgments and contrasts its performance with a Regex baseline. Using a ground-truth PoL corpus created by expert annotators on 60 judgments, the study shows ChatGPT struggles with precision, hallucinations, and inconsistent extraction, while a Regex-based approach—also aided by ChatGPT to generate the Regex—yields more PoLs and higher reliability. The results suggest AI lacks genuine understanding and integrated reasoning in this domain, though a human-guided, rule-based pipeline can provide effective tooling for legal experts. The work proposes a practical workflow for PoL extraction and outlines future directions, including broader datasets and evaluating additional language models and prompting techniques.

Abstract

This study examines the performance of ChatGPT with an experiment in the legal domain. We compare the outcome with it a baseline using regular expressions (Regex), rather than focusing solely on the assessment against human performance. The study reveals that even if ChatGPT has access to the necessary knowledge and competencies, it is unable to assemble them, reason through, in a way that leads to an exhaustive result. This unveils a major limitation of ChatGPT. Intelligence encompasses the ability to break down complex issues and address them according to multiple required competencies, providing a unified and comprehensive solution. In the legal domain, one of the most crucial tasks is reading legal decisions and extracting key passages condensed from principles of law (PoLs), which are then incorporated into subsequent rulings by judges or defense documents by lawyers. In performing this task, artificial intelligence lacks an all-encompassing understanding and reasoning, which makes it inherently limited. Genuine intelligence, remains a uniquely human trait, at least in this particular field.

ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate

TL;DR

The paper evaluates ChatGPT for extracting Principles of Law from Italian judgments and contrasts its performance with a Regex baseline. Using a ground-truth PoL corpus created by expert annotators on 60 judgments, the study shows ChatGPT struggles with precision, hallucinations, and inconsistent extraction, while a Regex-based approach—also aided by ChatGPT to generate the Regex—yields more PoLs and higher reliability. The results suggest AI lacks genuine understanding and integrated reasoning in this domain, though a human-guided, rule-based pipeline can provide effective tooling for legal experts. The work proposes a practical workflow for PoL extraction and outlines future directions, including broader datasets and evaluating additional language models and prompting techniques.

Abstract

This study examines the performance of ChatGPT with an experiment in the legal domain. We compare the outcome with it a baseline using regular expressions (Regex), rather than focusing solely on the assessment against human performance. The study reveals that even if ChatGPT has access to the necessary knowledge and competencies, it is unable to assemble them, reason through, in a way that leads to an exhaustive result. This unveils a major limitation of ChatGPT. Intelligence encompasses the ability to break down complex issues and address them according to multiple required competencies, providing a unified and comprehensive solution. In the legal domain, one of the most crucial tasks is reading legal decisions and extracting key passages condensed from principles of law (PoLs), which are then incorporated into subsequent rulings by judges or defense documents by lawyers. In performing this task, artificial intelligence lacks an all-encompassing understanding and reasoning, which makes it inherently limited. Genuine intelligence, remains a uniquely human trait, at least in this particular field.
Paper Structure (23 sections, 8 figures, 9 tables)

This paper contains 23 sections, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and Methodology
  3. Expert-Led Annotation Process
  4. Dataset
  5. Guidelines
  6. Limits
  7. Results
  8. Exploration with ChatGPT
  9. Discovering prompts
  10. Identifying PoLs
  11. Analysis and Results: Annotators vs ChatGPT
  12. Analysis
  13. Results
  14. Exploration with Regex
  15. Discovering Regex
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Excerpt of the annotation table tracking ChatGPT's PoLs extraction.
  • Figure 2: Comparison of the number of PoLs identified by the annotators versus those correctly detected by ChatGPT, with an average of 2.72, roundable up to 3.
  • Figure 3: Detailed comparison between the number of PoLs found by the annotators and by ChatGPT. Specifically, the comparison includes: the total number of correct PoLs found (labeled as "TOT ANN” for annotators and "TOT CHAT” for ChatGPT); implicit PoLs identified (labeled as "Implicit ANN” for annotators and "Implicit CHAT” for ChatGPT); explicit direct PoLs identified (labeled as "Ex. Direct ANN” for annotators and "Ex. Direct CHAT” for ChatGPT); explicit indirect PoLs identified (labeled as "Ex. Indirect ANN” for annotators and "Ex. Indirect CHAT” for ChatGPT). Then, hallucinations generated by ChatGPT and finally passages extracted as PoLs that were not actual PoLs (labeled as "Not-PoL CHAT").
  • Figure 4: Excerpt of the annotation table tracking Regex's PoLs extraction.
  • Figure 5: Comparison of the number of PoLs identified by the annotators versus those correctly identified by the Regex, with an average of 6.08.
  • ...and 3 more figures