A Benchmark for Lease Contract Review
Spyretta Leivaditi, Julien Rossi, Evangelos Kanoulas
TL;DR
The paper tackles automating lease contract review by defining and extracting two key elements: entities and red flags. It introduces a lease-focused benchmark dataset of 179 annotated lease agreements and presents ALeaseBERT, an ALBERT-based model pretrained on this data and fine-tuned for the two tasks. Results show that domain-specific pretraining improves performance for both red-flag detection (MAP up to 0.573) and entity extraction (weighted F1 up to ~0.54), though red-flag detection remains challenging at practical recall levels. These resources provide a solid baseline for future research and practical tooling to assist legal professionals in lease contract analysis.
Abstract
Extracting entities and other useful information from legal contracts is an important task whose automation can help legal professionals perform contract reviews more efficiently and reduce relevant risks. In this paper, we tackle the problem of detecting two different types of elements that play an important role in a contract review, namely entities and red flags. The latter are terms or sentences that indicate that there is some danger or other potentially problematic situation for one or more of the signing parties. We focus on supporting the review of lease agreements, a contract type that has received little attention in the legal information extraction literature, and we define the types of entities and red flags needed for that task. We release a new benchmark dataset of 179 lease agreement documents that we have manually annotated with the entities and red flags they contain, and which can be used to train and test relevant extraction algorithms. Finally, we release a new language model, called ALeaseBERT, pre-trained on this dataset and fine-tuned for the detection of the aforementioned elements, providing a baseline for further research