Explainability by design: an experimental analysis of the legal coding process
Matteo Cristani, Guido Governatori, Francesco Olivieri, Monica Palmirani, Gabriele Buriola
TL;DR
The paper investigates explainable legal coding by translating normative texts into Deontic Defeasible Logic (DDL) using Houdini, within a RegTech-inspired framework that aims to make the reasoning behind coded rules transparent. It presents a three-phase methodology (normative background encoding, scenario encoding, evaluation) and demonstrates it through illustrative translations and a human-in-the-loop experiment across Italian penal and private law texts. Key findings show coding is reliable with low error rates, but time per character varies considerably and decreases with expertise and reference depth; the study also quantifies substantial total effort for large codes and highlights language-induced variance as a critical challenge. The practical impact lies in establishing a measurable, explainable workflow for legal coding and in proposing directions (standardized token sets and corpus partitioning) to scale the process while preserving traceable reasoning.
Abstract
Behind a set of rules in Deontic Defeasible Logic, there is a mapping process of normative background fragments. This process goes from text to rules and implicitly encompasses an explanation of the coded fragments. In this paper we deliver a methodology for \textit{legal coding} that starts with a fragment and goes onto a set of Deontic Defeasible Logic rules, involving a set of \textit{scenarios} to test the correctness of the coded fragments. The methodology is illustrated by the coding process of an example text. We then show the results of a series of experiments conducted with humans encoding a variety of normative backgrounds and corresponding cases in which we have measured the efforts made in the coding process, as related to some measurable features. To process these examples, a recently developed technology, Houdini, that allows reasoning in Deontic Defeasible Logic, has been employed. Finally we provide a technique to forecast time required in coding, that depends on factors such as knowledge of the legal domain, knowledge of the coding processes, length of the text, and a measure of \textit{depth} that refers to the length of the paths of legal references.