Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laypeople

Shounak Paul, Raghav Dogra, Pawan Goyal, Saptarshi Ghosh

TL;DR

This work addresses the problem of identifying Indian statutes from informal layperson queries and introduces two corpora, ILSIC-Lay and ILSIC-Multi, to enable direct comparison between layperson queries and court facts under a common statute space. It benchmarks a broad spectrum of retrieval and generative approaches, including BM25, SBERT, SAILER, GPT-4.1, Llama-3, Gemma-3, Qwen-3, and Phi-4 Mini, across zero-shot, few-shot, RAG, and supervised fine-tuning settings, with a Verbalization pipeline to canonical statute formats. The results reveal the substantial gap between court-trained models and lay query language, the limited but sometimes beneficial effects of transfer learning from court data, and the strong generalization capabilities of large LLMs like GPT-4.1 in zero-shot settings, while highlighting the bottlenecks in retrieval-based augmentation. The study provides practical guidance for building layperson-oriented legal AI tools in India and points to future work on multilingual, richer-context data and improved data-fusion strategies to bridge the gap between formal and informal legal language.

Abstract

Legal Statute Identification (LSI) for a given situation is one of the most fundamental tasks in Legal NLP. This task has traditionally been modeled using facts from court judgments as input queries, due to their abundance. However, in practical settings, the input queries are likely to be informal and asked by laypersons, or non-professionals. While a few laypeople LSI datasets exist, there has been little research to explore the differences between court and laypeople data for LSI. In this work, we create ILSIC, a corpus of laypeople queries covering 500+ statutes from Indian law. Additionally, the corpus also contains court case judgements to enable researchers to effectively compare between court and laypeople data for LSI. We conducted extensive experiments on our corpus, including benchmarking over the laypeople dataset using zero and few-shot inference, retrieval-augmented generation and supervised fine-tuning. We observe that models trained purely on court judgements are ineffective during test on laypeople queries, while transfer learning from court to laypeople data can be beneficial in certain scenarios. We also conducted fine-grained analyses of our results in terms of categories of queries and frequency of statutes.

ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laypeople

TL;DR

This work addresses the problem of identifying Indian statutes from informal layperson queries and introduces two corpora, ILSIC-Lay and ILSIC-Multi, to enable direct comparison between layperson queries and court facts under a common statute space. It benchmarks a broad spectrum of retrieval and generative approaches, including BM25, SBERT, SAILER, GPT-4.1, Llama-3, Gemma-3, Qwen-3, and Phi-4 Mini, across zero-shot, few-shot, RAG, and supervised fine-tuning settings, with a Verbalization pipeline to canonical statute formats. The results reveal the substantial gap between court-trained models and lay query language, the limited but sometimes beneficial effects of transfer learning from court data, and the strong generalization capabilities of large LLMs like GPT-4.1 in zero-shot settings, while highlighting the bottlenecks in retrieval-based augmentation. The study provides practical guidance for building layperson-oriented legal AI tools in India and points to future work on multilingual, richer-context data and improved data-fusion strategies to bridge the gap between formal and informal legal language.

Abstract

Legal Statute Identification (LSI) for a given situation is one of the most fundamental tasks in Legal NLP. This task has traditionally been modeled using facts from court judgments as input queries, due to their abundance. However, in practical settings, the input queries are likely to be informal and asked by laypersons, or non-professionals. While a few laypeople LSI datasets exist, there has been little research to explore the differences between court and laypeople data for LSI. In this work, we create ILSIC, a corpus of laypeople queries covering 500+ statutes from Indian law. Additionally, the corpus also contains court case judgements to enable researchers to effectively compare between court and laypeople data for LSI. We conducted extensive experiments on our corpus, including benchmarking over the laypeople dataset using zero and few-shot inference, retrieval-augmented generation and supervised fine-tuning. We observe that models trained purely on court judgements are ineffective during test on laypeople queries, while transfer learning from court to laypeople data can be beneficial in certain scenarios. We also conducted fine-grained analyses of our results in terms of categories of queries and frequency of statutes.
Paper Structure (27 sections, 4 figures, 20 tables)

This paper contains 27 sections, 4 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Corpora for Legal Statute Identification
  4. Construction of ILSIC-Lay
  5. Construction of ILSIC-Multi
  6. Methods and General Experimental Settings
  7. Retrieval-based Methods
  8. Generative Methods
  9. Evaluation Metrics
  10. Experiments on ILSIC-Lay
  11. Results
  12. Analysis
  13. Evaluation with Human Experts
  14. Experiments on ILSIC-Multi
  15. Results
  16. ...and 12 more sections

Figures (4)

  • Figure 1: F1 Scores vs query categories sorted according to frequency over ILSIC-Lay$_{test}$ (left to right in descending order of category frequency)
  • Figure 2: F1 Scores vs Statutes sorted according to frequency over ILSIC-Lay$_{test}$ and divided into 10 groups (left to right in descending order of statute frequency)
  • Figure 3: Comparison of fine-tuning models across different settings and categories of queries
  • Figure 4: Comparison of fine-tuning models across different settings and frequency of statutes