Intelligent Legal Assistant: An Interactive Clarification System for Legal Question Answering

TL;DR

The paper tackles information-deficient user questions in legal Q&A by introducing Intelligent Legal Assistant, which adds geographic localization and an interactive clarification loop to elicit the missing details needed for accurate advice. The approach combines deficiency detection, IRAC-based key node graph construction, RL-driven missing-node prediction with graph neural network embeddings, clarifying-question generation via in-context learning, and retrieval-augmented final answers synthesized by a large language model. The RL objective for policy optimization is $J(\theta) = \mathbb{E}[\sum_{t=0}^{\infty} (\gamma^t (r_t + \gamma V_\phi(s_{t+1}) - V_\phi(s_t)))]$, enabling long-horizon planning for information completeness. In a blind evaluation with 100 users, the system outperformed GPT-4o, AI Lawyer, and Callidus AI on accuracy, satisfaction, and user preference, demonstrating practical gains in region-specific, precise legal guidance and highlighting ethical data-use practices for user studies.

Abstract

The rise of large language models has opened new avenues for users seeking legal advice. However, users often lack professional legal knowledge, which can lead to questions that omit critical information. This deficiency makes it challenging for traditional legal question-answering systems to accurately identify users' actual needs, often resulting in imprecise or generalized advice. In this work, we develop a legal question-answering system called Intelligent Legal Assistant, which interacts with users to precisely capture their needs. When a user poses a question, the system requests that the user select their geographical location to pinpoint the applicable laws. It then generates clarifying questions and options based on the key information missing from the user's initial question. This allows the user to select and provide the necessary details. Once all necessary information is provided, the system produces an in-depth legal analysis encompassing three aspects: overall conclusion, jurisprudential analysis, and resolution suggestions.

Figures (3)

• Figure 1: Comparison of our system with traditional LLM-based Q&A systems. ①-⑤ are the steps of our system, and I-III are the steps of traditional LLM-based Q&A systems.
• Figure 2: System architecture. The system consists of three primary functional modules: 1) information deficiency detection, 2) clarifying question and option generation, and 3) comprehensive response generation.
• Figure 3: System Demo with User-LLM Interactions.