Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeliLaw: A Chinese Legal Counselling System Based on a Large Language Model

Nan Xie, Yuelin Bai, Hengyuan Gao, Feiteng Fang, Qixuan Zhao, Zhijian Li, Ziqiang Xue, Liang Zhu, Shiwen Ni, Min Yang

TL;DR

DeliLaw addresses the challenge of semantically rich, domain-specific Chinese legal queries by integrating a law-focused retrieval stack with a fine-tuned large language model to deliver dialogue-based legal counselling. It combines a Law Retriever, Case Retriever, and a law-specific LLM trained with diverse legal data sources and augmented content, using two-stage fine-tuning and retrieval to reduce hallucinations. The system leverages Milvus for embedding-based statute retrieval, ElasticSearch for case search, and a RoBERTa-based intent classifier to route queries, achieving strong retrieval metrics and professional, legally-grounded responses. Real-world evaluation shows improved accuracy, citation quality, and logical consistency, with substantial user adoption (over 20,000 professional users and ~3,000 calls daily), and ongoing work to broaden coverage and enhance contextual understanding and user experience.

Abstract

Traditional legal retrieval systems designed to retrieve legal documents, statutes, precedents, and other legal information are unable to give satisfactory answers due to lack of semantic understanding of specific questions. Large Language Models (LLMs) have achieved excellent results in a variety of natural language processing tasks, which inspired us that we train a LLM in the legal domain to help legal retrieval. However, in the Chinese legal domain, due to the complexity of legal questions and the rigour of legal articles, there is no legal large model with satisfactory practical application yet. In this paper, we present DeliLaw, a Chinese legal counselling system based on a large language model. DeliLaw integrates a legal retrieval module and a case retrieval module to overcome the model hallucination. Users can consult professional legal questions, search for legal articles and relevant judgement cases, etc. on the DeliLaw system in a dialogue mode. In addition, DeliLaw supports the use of English for counseling. we provide the address of the system: https://data.delilegal.com/lawQuestion.

DeliLaw: A Chinese Legal Counselling System Based on a Large Language Model

TL;DR

DeliLaw addresses the challenge of semantically rich, domain-specific Chinese legal queries by integrating a law-focused retrieval stack with a fine-tuned large language model to deliver dialogue-based legal counselling. It combines a Law Retriever, Case Retriever, and a law-specific LLM trained with diverse legal data sources and augmented content, using two-stage fine-tuning and retrieval to reduce hallucinations. The system leverages Milvus for embedding-based statute retrieval, ElasticSearch for case search, and a RoBERTa-based intent classifier to route queries, achieving strong retrieval metrics and professional, legally-grounded responses. Real-world evaluation shows improved accuracy, citation quality, and logical consistency, with substantial user adoption (over 20,000 professional users and ~3,000 calls daily), and ongoing work to broaden coverage and enhance contextual understanding and user experience.

Abstract

Traditional legal retrieval systems designed to retrieve legal documents, statutes, precedents, and other legal information are unable to give satisfactory answers due to lack of semantic understanding of specific questions. Large Language Models (LLMs) have achieved excellent results in a variety of natural language processing tasks, which inspired us that we train a LLM in the legal domain to help legal retrieval. However, in the Chinese legal domain, due to the complexity of legal questions and the rigour of legal articles, there is no legal large model with satisfactory practical application yet. In this paper, we present DeliLaw, a Chinese legal counselling system based on a large language model. DeliLaw integrates a legal retrieval module and a case retrieval module to overcome the model hallucination. Users can consult professional legal questions, search for legal articles and relevant judgement cases, etc. on the DeliLaw system in a dialogue mode. In addition, DeliLaw supports the use of English for counseling. we provide the address of the system: https://data.delilegal.com/lawQuestion.
Paper Structure (16 sections, 3 figures)

This paper contains 16 sections, 3 figures.

Table of Contents

  1. Introduction
  2. System Overview
  3. Law Retriever
  4. Case Retriever
  5. Law-specific Fine-tuned LLM
  6. Inference Process
  7. DeliLaw Datasets
  8. Open General Data
  9. Legal Data
  10. Legal Q&A data
  11. Human annotated data
  12. Cause of actions in GPT-4
  13. Judicial syllogism Data
  14. Guided conversations
  15. Evaluation
  16. ...and 1 more sections

Figures (3)

  • Figure 1: The interface of DeliLaw system.
  • Figure 2: An illustration of DeliLaw's inference process.
  • Figure 3: Human assessment, comparing DeliLaw to 4 different baselines across 100 test questions.