InternLM-Law: An Open Source Chinese Legal Large Language Model

TL;DR

This paper introduces InternLM-Law, a specialized LLM tailored for addressing diverse legal queries related to Chinese laws, spanning from responding to standard legal questions to analyzing complex real-world legal situations.

Abstract

While large language models (LLMs) have showcased impressive capabilities, they struggle with addressing legal queries due to the intricate complexities and specialized expertise required in the legal field. In this paper, we introduce InternLM-Law, a specialized LLM tailored for addressing diverse legal queries related to Chinese laws, spanning from responding to standard legal questions (e.g., legal exercises in textbooks) to analyzing complex real-world legal situations. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries, and implement a data filtering and processing pipeline to ensure its diversity and quality. Our training approach involves a novel two-stage process: initially fine-tuning LLMs on both legal-specific and general-purpose content to equip the models with broad knowledge, followed by exclusive fine-tuning on high-quality legal data to enhance structured output generation. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks. We make InternLM-Law and our dataset publicly available to facilitate future research in applying LLMs within the legal domain.

Figures (7)

• Figure 1: Model performance (zero-shot) evaluated across 20 legal subtasks from LawBench. Our proposed model, InternLM-Law-7B, achives the highest overall performance, outperforming GPT-4 and other larger-scale Chinese general-domain model such as Qwen-72b.
• Figure 2: Illustration of our two-stage training pipeline. Initially, InternLM2-chat is trained on a diverse range of tasks including both general-purpose and legal tasks. Subsequently, it is further trained specifically on high-quality legal tasks.
• Figure 3: Data processing pipeline for legal consultation data. Initial filtering of answers is conducted using rule-based methods, such as regular expressions. Subsequently, the quality assessment of questions and the coherence between questions and answers is performed utilizing LLMs.
• Figure 4: We manually create a set of seed instructions for each task, which are subsequently expanded using GPT-4 through paraphrasing, resulting in a rich pool of diverse instructions. During training, each training instance is paired with an instruction randomly chosen from this pool.
• Figure 5: An example of legal consultation.