A document processing pipeline for the construction of a dataset for topic modeling based on the judgments of the Italian Supreme Court
Matteo Marulli, Glauco Panattoni, Marco Bertini
TL;DR
This work tackles the absence of public Italian legal datasets suitable for topic modeling by introducing an end-to-end document processing pipeline that converts Supreme Court judgments into an anonymized, NLP-ready dataset. The pipeline fuses document layout analysis with OCR and GDPR compliant anonymization, yielding robust DLA and OCR performance and enabling more diverse and coherent topic models via BERTopic. It further explores topic interpretation with large language models, comparing outputs against domain expert annotations to assess reliability. The results demonstrate improved topic modeling quality and provide practical resources for Italian legal NLP, while highlighting limitations in anonymization evaluation and suggesting directions for future expansion to other document types and jurisdictions.
Abstract
Topic modeling in Italian legal research is hindered by the lack of public datasets, limiting the analysis of legal themes in Supreme Court judgments. To address this, we developed a document processing pipeline that produces an anonymized dataset optimized for topic modeling. The pipeline integrates document layout analysis (YOLOv8x), optical character recognition, and text anonymization. The DLA module achieved a mAP@50 of 0.964 and a mAP@50-95 of 0.800. The OCR detector reached a mAP@50-95 of 0.9022, and the text recognizer (TrOCR) obtained a character error rate of 0.0047 and a word error rate of 0.0248. Compared to OCR-only methods, our dataset improved topic modeling with a diversity score of 0.6198 and a coherence score of 0.6638. We applied BERTopic to extract topics and used large language models to generate labels and summaries. Outputs were evaluated against domain expert interpretations. Claude Sonnet 3.7 achieved a BERTScore F1 of 0.8119 for labeling and 0.9130 for summarization.