The 3rd Place Solution of CCIR CUP 2025: A Framework for Retrieval-Augmented Generation in Multi-Turn Legal Conversation

TL;DR

This work develops a retrieval-augmented generation framework tailored for multi-turn legal conversations, combining query rewriting, dense-sparse multi-route retrieval, legal literature filtering, and reranked generation to ground responses in retrieved statutes. It demonstrates that modular enhancements—especially query rewriting and literature filtering—yield systematic gains in both retrieval quality and answer factuality, with further improvements from a dedicated reranker and hybrid retrieval. The approach achieves competitive performance on CCIR CUP 2025 (3rd place) and provides a comprehensive analysis, including ablation studies, component analyses, case studies, and error analysis. Overall, the framework advances practical, scalable, and legally grounded conversational AI for knowledge-intensive domains while emphasizing responsible deployment and clear citations of legal provisions.

Abstract

Retrieval-Augmented Generation has made significant progress in the field of natural language processing. By combining the advantages of information retrieval and large language models, RAG can generate relevant and contextually appropriate responses based on items retrieved from reliable sources. This technology has demonstrated outstanding performance across multiple domains, but its application in the legal field remains in its exploratory phase. In this paper, we introduce our approach for "Legal Knowledge Retrieval and Generation" in CCIR CUP 2025, which leverages large language models and information retrieval systems to provide responses based on laws in response to user questions.

Figures (6)

• Figure 1: The pipeline of our method.
• Figure 2: The prompt of query rewriting. The italicized text is the output of the LLM.
• Figure 3: The prompt of literature selection. The italicized text is the output of the LLM.
• Figure 4: The prompt of filtering. The italicized text is the output of the LLM.
• Figure 5: The prompt of response generation. The italicized text is the output of the LLM.