Improving Legal Case Retrieval with Brain Signals

TL;DR

This work tackles the challenge of relevance feedback in legal case retrieval by introducing a brain-signal–based framework that harvests EEG data during long-form document reading. It develops a long-duration EEG feature extraction pipeline and uses SVM-RFE to predict per-paragraph usefulness, followed by a topic-intent based re-ranking strategy. The study demonstrates that EEG-driven feedback can outperform traditional cues (clicks or none) in terms of accuracy, F1, and user satisfaction, validated on a LeCard-derived task set with 20 domain experts and an accompanying public EEG dataset. The approach promises seamless, fine-grained relevance feedback for professional legal search and provides a foundation for real-time, EEG-informed retrieval systems.

Abstract

The tasks of legal case retrieval have received growing attention from the IR community in the last decade. Relevance feedback techniques with implicit user feedback (e.g., clicks) have been demonstrated to be effective in traditional search tasks (e.g., Web search). In legal case retrieval, however, collecting relevance feedback faces a couple of challenges that are difficult to resolve under existing feedback paradigms. First, legal case retrieval is a complex task as users often need to understand the relationship between legal cases in detail to correctly judge their relevance. Traditional feedback signal such as clicks is too coarse to use as they do not reflect any fine-grained relevance information. Second, legal case documents are usually long, users often need even tens of minutes to read and understand them. Simple behavior signal such as clicks and eye-tracking fixations can hardly be useful when users almost click and examine every part of the document. In this paper, we explore the possibility of solving the feedback problem in legal case retrieval with brain signal. Recent advances in brain signal processing have shown that human emotional can be collected in fine grains through Brain-Machine Interfaces (BMI) without interrupting the users in their tasks. Therefore, we propose a framework for legal case retrieval that uses EEG signal to optimize retrieval results. We collected and create a legal case retrieval dataset with users EEG signal and propose several methods to extract effective EEG features for relevance feedback. Our proposed features achieve a 71% accuracy for feedback prediction with an SVM-RFE model, and our proposed ranking method that takes into account the diverse needs of users can significantly improve user satisfaction for legal case retrieval. Experiment results show that re-ranked result list make user more satisfied.

Figures (5)

• Figure 1: An example legal case in our corpus. A legal case composes of four parts: Basic information, Case description, Court analysis and Judgement result.
• Figure 2: Legal case retrieval system supported by brain signals. The user inputs the description of her task case to begin the search process. Blue parts show the user's procedure when using this framework. Red parts shows that during her judgments, EEG signals are collected and analyzed by the system. Based on the the analysis result, the system provides re-ranked results to the user.
• Figure 3: Example of the page in which users view the judgment (translated).
• Figure 4: Comparison of Event-Related-Potential (ERP) in Web search scenario and ERP in leagl search scenario.
• Figure 5: Experimental process. Blue lines shows the process of the participants' interactions. Red lines shows the sub-process in S2. Green lines are the data flow of annotations. S2 stands for a loop of reading paragraphs of single judgement, while S2-S3-S4 stands for a large loop of reading multiple judgements of a single task. The process of S1-S2-S3-S4-S5 stands for a tasks' process and will repeat 4 times for each participant. First, participants read the task description (S1). Second, participants read the first judgment given by the system (S2). During the reading process, subjects could choose to read one of the paragraphs each time. And they are allowed to end the reading if they think enough information is acquired. Third, participants annotated the judgment they had just read (S3). Fourth, participants choose whether to read next judgment (S4). Fifth, participants mark the completion of the whole task (S5).