Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HLDC: Hindi Legal Documents Corpus

Arnav Kapoor, Mudit Dhawan, Anmol Goel, T. H. Arjun, Akshala Bhatnagar, Vibhu Agrawal, Amul Agrawal, Arnab Bhattacharya, Ponnurangam Kumaraguru, Ashutosh Modi

TL;DR

HLDC addresses the shortage of Hindi legal corpora by releasing HLDC, a dataset of 912,568 Hindi district-court documents from Uttar Pradesh, cleaned, anonymized, and semi-structured for NLP tasks. It defines Bail Prediction as a binary classifier on document facts $p_{\\theta}(y|f)$ and evaluates multiple models, including an end-to-end Multi-Task Learning (MTL) model that uses an auxiliary salience task and achieves about 78.5% accuracy on district-wise splits with an AUC of 0.85. The results underscore the value of extracting salient content from long, unstructured legal texts and show that district-level lexical variation can affect generalization. The authors release HLDC and code to foster broader research and plan to extend the corpus to more states and languages, enabling future tasks like summarization and prior-case retrieval.

Abstract

Many populous countries including India are burdened with a considerable backlog of legal cases. Development of automated systems that could process legal documents and augment legal practitioners can mitigate this. However, there is a dearth of high-quality corpora that is needed to develop such data-driven systems. The problem gets even more pronounced in the case of low resource languages such as Hindi. In this resource paper, we introduce the Hindi Legal Documents Corpus (HLDC), a corpus of more than 900K legal documents in Hindi. Documents are cleaned and structured to enable the development of downstream applications. Further, as a use-case for the corpus, we introduce the task of bail prediction. We experiment with a battery of models and propose a Multi-Task Learning (MTL) based model for the same. MTL models use summarization as an auxiliary task along with bail prediction as the main task. Experiments with different models are indicative of the need for further research in this area. We release the corpus and model implementation code with this paper: https://github.com/Exploration-Lab/HLDC

HLDC: Hindi Legal Documents Corpus

TL;DR

HLDC addresses the shortage of Hindi legal corpora by releasing HLDC, a dataset of 912,568 Hindi district-court documents from Uttar Pradesh, cleaned, anonymized, and semi-structured for NLP tasks. It defines Bail Prediction as a binary classifier on document facts and evaluates multiple models, including an end-to-end Multi-Task Learning (MTL) model that uses an auxiliary salience task and achieves about 78.5% accuracy on district-wise splits with an AUC of 0.85. The results underscore the value of extracting salient content from long, unstructured legal texts and show that district-level lexical variation can affect generalization. The authors release HLDC and code to foster broader research and plan to extend the corpus to more states and languages, enabling future tasks like summarization and prior-case retrieval.

Abstract

Many populous countries including India are burdened with a considerable backlog of legal cases. Development of automated systems that could process legal documents and augment legal practitioners can mitigate this. However, there is a dearth of high-quality corpora that is needed to develop such data-driven systems. The problem gets even more pronounced in the case of low resource languages such as Hindi. In this resource paper, we introduce the Hindi Legal Documents Corpus (HLDC), a corpus of more than 900K legal documents in Hindi. Documents are cleaned and structured to enable the development of downstream applications. Further, as a use-case for the corpus, we introduce the task of bail prediction. We experiment with a battery of models and propose a Multi-Task Learning (MTL) based model for the same. MTL models use summarization as an auxiliary task along with bail prediction as the main task. Experiments with different models are indicative of the need for further research in this area. We release the corpus and model implementation code with this paper: https://github.com/Exploration-Lab/HLDC
Paper Structure (33 sections, 3 equations, 7 figures, 9 tables)

This paper contains 33 sections, 3 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hindi Legal Documents Corpus
  4. HLDC: Ethical Aspects
  5. Bail Prediction Task
  6. Bail Prediction Models
  7. Embedding Based Models
  8. Summarization Based Models
  9. Multi-Task Learning (MTL) Model
  10. Bail Prediction Task
  11. Salience Classification Task
  12. Experiments and Results
  13. Dataset Splits
  14. Results
  15. Error Analysis
  16. ...and 18 more sections

Figures (7)

  • Figure 1: HLDC corpus creation pipeline
  • Figure 2: Variation in number of case documents per district in the state of U.P. Prominent districts are marked.
  • Figure 3: Ratio of number of bail applications to total number of applications in U.P.
  • Figure 4: Bail Corpus Creation Pipeline
  • Figure 5: Overview of our multi-task learning approach.
  • ...and 2 more figures