Mitigating Manipulation and Enhancing Persuasion: A Reflective Multi-Agent Approach for Legal Argument Generation
Li Zhang, Kevin D. Ashley
TL;DR
The paper addresses manipulation risks and grounding gaps in LLM-based legal argumentation and introduces a Reflective Multi-Agent framework with a Factor Analyst and an Argument Polisher to generate a 3-ply structure (Plaintiff's Argument, Defendant's Counterargument, Plaintiff's Rebuttal). It evaluates across four LLMs and three case scenarios using metrics such as $Acc_H$, $Rec_U$, and $Ratio_{Abstain}$, demonstrating that reflective, agent-based refinement reduces intrinsic hallucination and improves abstention when grounding is lacking. Results show that the Reflective Multi-Agent framework achieves higher grounding quality and abstention rates while maintaining strong factual fidelity across models, particularly in challenging Non-Arguable cases. The findings suggest that structured reflection within a multi-agent system yields more ethically persuasive, legally grounded AI, with future work exploring external knowledge grounding, full-text document processing, and human-in-the-loop refinements to further enhance reliability and fairness in legal AI tools.
Abstract
Large Language Models (LLMs) are increasingly explored for legal argument generation, yet they pose significant risks of manipulation through hallucination and ungrounded persuasion, and often fail to utilize provided factual bases effectively or abstain when arguments are untenable. This paper introduces a novel reflective multi-agent method designed to address these challenges in the context of legally compliant persuasion. Our approach employs specialized agents (factor analyst and argument polisher) in an iterative refinement process to generate 3-ply legal arguments (plaintiff, defendant, rebuttal). We evaluate reflective multi-agent against single-agent, enhanced-prompt single-agent, and non-reflective multi-agent baselines using four diverse LLMs (GPT-4o, GPT-4o-mini, Llama-4-Maverick-17b-128e, Llama-4-Scout-17b-16e) across three legal scenarios: "arguable", "mismatched", and "non-arguable". Results demonstrate that the reflective multi-agent approach excels at successful abstention by preventing generation when arguments cannot be grounded, improves hallucination accuracy by reducing fabricated and misattributed factors and enhances factor utilization recall by better using the provided case facts. These findings suggest that structured reflection within a multi-agent framework offers a robust method for fostering ethical persuasion and mitigating manipulation in LLM-based legal argumentation systems.