Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model

Zhi Zhou, Jiang-Xin Shi, Peng-Xiao Song, Xiao-Wen Yang, Yi-Xuan Jin, Lan-Zhe Guo, Yu-Feng Li

TL;DR

The paper addresses the privacy and knowledge gaps of existing LLMs in Chinese legal tasks by introducing LawGPT, the first open-source Chinese legal knowledge-enhanced LLM. It employs a two-stage training pipeline: legal-oriented pre-training on a 500K legal document corpus to embed domain knowledge and legal supervised fine-tuning on a 300K knowledge-driven instruction set to align outputs with legal tasks. Empirical results show that LawGPT surpasses the open-source LLaMA-7B baseline on major legal tasks, while still lagging behind proprietary models, underscoring a trade-off between privacy and raw performance. The work provides a practical, privacy-forward alternative for legal AI with publicly available code and resources, potentially catalyzing further improvements in Chinese legal AI.

Abstract

Large language models (LLMs), including both proprietary and open-source models, have showcased remarkable capabilities in addressing a wide range of downstream tasks. Nonetheless, when it comes to practical Chinese legal tasks, these models fail to meet the actual requirements. Proprietary models do not ensure data privacy for sensitive legal cases, while open-source models demonstrate unsatisfactory performance due to their lack of legal knowledge. To address this problem, we introduce LawGPT, the first open-source model specifically designed for Chinese legal applications. LawGPT comprises two key components: legal-oriented pre-training and legal supervised fine-tuning. Specifically, we employ large-scale Chinese legal documents for legal-oriented pre-training to incorporate legal domain knowledge. To further improve the model's performance on downstream legal tasks, we create a knowledge-driven instruction dataset for legal supervised fine-tuning. Our experimental results demonstrate that LawGPT outperforms the open-source LLaMA 7B model. Our code and resources are publicly available at https://github.com/pengxiao-song/LaWGPT and have received 5.7K stars on GitHub.

LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model

TL;DR

The paper addresses the privacy and knowledge gaps of existing LLMs in Chinese legal tasks by introducing LawGPT, the first open-source Chinese legal knowledge-enhanced LLM. It employs a two-stage training pipeline: legal-oriented pre-training on a 500K legal document corpus to embed domain knowledge and legal supervised fine-tuning on a 300K knowledge-driven instruction set to align outputs with legal tasks. Empirical results show that LawGPT surpasses the open-source LLaMA-7B baseline on major legal tasks, while still lagging behind proprietary models, underscoring a trade-off between privacy and raw performance. The work provides a practical, privacy-forward alternative for legal AI with publicly available code and resources, potentially catalyzing further improvements in Chinese legal AI.

Abstract

Large language models (LLMs), including both proprietary and open-source models, have showcased remarkable capabilities in addressing a wide range of downstream tasks. Nonetheless, when it comes to practical Chinese legal tasks, these models fail to meet the actual requirements. Proprietary models do not ensure data privacy for sensitive legal cases, while open-source models demonstrate unsatisfactory performance due to their lack of legal knowledge. To address this problem, we introduce LawGPT, the first open-source model specifically designed for Chinese legal applications. LawGPT comprises two key components: legal-oriented pre-training and legal supervised fine-tuning. Specifically, we employ large-scale Chinese legal documents for legal-oriented pre-training to incorporate legal domain knowledge. To further improve the model's performance on downstream legal tasks, we create a knowledge-driven instruction dataset for legal supervised fine-tuning. Our experimental results demonstrate that LawGPT outperforms the open-source LLaMA 7B model. Our code and resources are publicly available at https://github.com/pengxiao-song/LaWGPT and have received 5.7K stars on GitHub.
Paper Structure (13 sections, 2 equations, 6 figures, 1 table)

This paper contains 13 sections, 2 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. General Language Models
  4. Legal Language Models
  5. Legal Benchmarks
  6. Methodology
  7. Legal-Oriented Pre-Training
  8. Legal-Supervised Fine-Tuning
  9. Inference of LawGPT
  10. Experiments
  11. Implementation Details
  12. Performance Evaluation
  13. Conclusion