Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Legal Assist AI: Leveraging Transformer-Based Model for Effective Legal Assistance

Jatin Gupta, Akhil Sharma, Saransh Singhania, Ali Imam Abidi

TL;DR

This work tackles limited access to Indian legal information by building Legal Assist AI, a retrieval-augmented transformer tailored to Indian law. It uses a base Llama 3.1 8B, retrieval from a FAISS vector store built from Indian constitutional and statutory texts, and a prompt system to maintain legal accuracy. The system achieves 60.08% on the All India Bar Examination, outperforming GPT-3.5 Turbo, Mistral 7B, and AALAP, while reducing hallucinations relative to these baselines. The results demonstrate the viability of domain-specific, retrieval-augmented LLMs for practical multilingual legal assistance and lay groundwork for expanding datasets and language coverage in the Indian legal domain.

Abstract

Pursuit of accessible legal assistance in India faces a critical gap, as many citizens struggle to leverage their legal rights due to limited awareness and access to relevant legal information. This paper introduces Legal Assist AI, a transformer-based model designed to bridge this gap by offering effective legal assistance through large language models (LLMs). The system retrieves relevant legal information from a curated database and generates accurate responses, enabling effective assistance for diverse users, including legal professionals, scholars, and the general public. The model was fine-tuned on extensive datasets from the Indian legal domain, including Indian Constitution, Bharatiya Nyaya Sanhita (BNS), Bharatiya Nagarik Suraksha Sanhita (BNSS) and so forth, providing a robust understanding of the complexities of Indian law. By incorporating domain-specific legal datasets, the proposed model demonstrated remarkable efficiency and specialization in legal Question-Answering. The model was evaluated against state-of-the-art models such as GPT-3.5 Turbo and Mistral 7B, achieving a 60.08% score on the AIBE, outperforming its competitors in legal reasoning and accuracy. Unlike other models, Legal Assist AI avoided common issues such as hallucinations, making it highly reliable for practical legal applications. It showcases the model's applicability in real-world legal scenarios, with future iterations aiming to enhance performance and expand its dataset to cover a broader range of multilingual and case-specific queries as well.

Legal Assist AI: Leveraging Transformer-Based Model for Effective Legal Assistance

TL;DR

This work tackles limited access to Indian legal information by building Legal Assist AI, a retrieval-augmented transformer tailored to Indian law. It uses a base Llama 3.1 8B, retrieval from a FAISS vector store built from Indian constitutional and statutory texts, and a prompt system to maintain legal accuracy. The system achieves 60.08% on the All India Bar Examination, outperforming GPT-3.5 Turbo, Mistral 7B, and AALAP, while reducing hallucinations relative to these baselines. The results demonstrate the viability of domain-specific, retrieval-augmented LLMs for practical multilingual legal assistance and lay groundwork for expanding datasets and language coverage in the Indian legal domain.

Abstract

Pursuit of accessible legal assistance in India faces a critical gap, as many citizens struggle to leverage their legal rights due to limited awareness and access to relevant legal information. This paper introduces Legal Assist AI, a transformer-based model designed to bridge this gap by offering effective legal assistance through large language models (LLMs). The system retrieves relevant legal information from a curated database and generates accurate responses, enabling effective assistance for diverse users, including legal professionals, scholars, and the general public. The model was fine-tuned on extensive datasets from the Indian legal domain, including Indian Constitution, Bharatiya Nyaya Sanhita (BNS), Bharatiya Nagarik Suraksha Sanhita (BNSS) and so forth, providing a robust understanding of the complexities of Indian law. By incorporating domain-specific legal datasets, the proposed model demonstrated remarkable efficiency and specialization in legal Question-Answering. The model was evaluated against state-of-the-art models such as GPT-3.5 Turbo and Mistral 7B, achieving a 60.08% score on the AIBE, outperforming its competitors in legal reasoning and accuracy. Unlike other models, Legal Assist AI avoided common issues such as hallucinations, making it highly reliable for practical legal applications. It showcases the model's applicability in real-world legal scenarios, with future iterations aiming to enhance performance and expand its dataset to cover a broader range of multilingual and case-specific queries as well.

Paper Structure

This paper contains 19 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Data Preparation
  5. Proposed Framework
  6. Model Selection and Architecture
  7. Prompt
  8. Training
  9. Model Evaluations
  10. Evaluation using Lawyer_GPT_India dataset
  11. Dataset
  12. Observations
  13. Evaluation using All India Bar Examination dataset
  14. Dataset
  15. Observations
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Architecture of Base Model Used
  • Figure 2: Proposed Legal Assist AI Architecture
  • Figure 3: System Prompt