Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLaMandement: Large Language Models for Summarization of French Legislative Proposals

Joseph Gesnouin, Yannis Tannier, Christophe Gomes Da Silva, Hatim Tapory, Camille Brier, Hugo Simon, Raphael Rozenberg, Hermann Woehrel, Mehdi El Yakaabi, Thomas Binder, Guillaume Marie, Emilie Caron, Mathile Nogueira, Thomas Fontas, Laure Puydebois, Marie Theophile, Stephane Morandi, Mael Petit, David Creissac, Pauline Ennouchy, Elise Valetoux, Celine Visade, Severine Balloux, Emmanuel Cortes, Pierre-Etienne Devineau, Ulrich Tan, Esther Mac Namara, Su Yang

TL;DR

The paper addresses the growing burden of processing amendments in the French Parliament and the need for neutral, scalable summaries to aid interministerial coordination. It introduces LLaMandement, a LLaMA-based system fine-tuned with LoRA on bench memoranda and amendment summaries from SIGNALEdila, to produce neutral, concise summaries. Through human evaluator ratings by fiscal drafters and bias assessments using the BOLD dataset, it demonstrates near-human performance for summarization and shows only minimal bias relative to baselines. The authors publish fine-tuned weights and training data to public commons, highlighting practical impact for automated legislative analysis and transparent government AI.

Abstract

This report introduces LLaMandement, a state-of-the-art Large Language Model, fine-tuned by the French government and designed to enhance the efficiency and efficacy of processing parliamentary sessions (including the production of bench memoranda and documents required for interministerial meetings) by generating neutral summaries of legislative proposals. Addressing the administrative challenges of manually processing a growing volume of legislative amendments, LLaMandement stands as a significant legal technological milestone, providing a solution that exceeds the scalability of traditional human efforts while matching the robustness of a specialized legal drafter. We release all our fine-tuned models and training data to the community.

LLaMandement: Large Language Models for Summarization of French Legislative Proposals

TL;DR

The paper addresses the growing burden of processing amendments in the French Parliament and the need for neutral, scalable summaries to aid interministerial coordination. It introduces LLaMandement, a LLaMA-based system fine-tuned with LoRA on bench memoranda and amendment summaries from SIGNALEdila, to produce neutral, concise summaries. Through human evaluator ratings by fiscal drafters and bias assessments using the BOLD dataset, it demonstrates near-human performance for summarization and shows only minimal bias relative to baselines. The authors publish fine-tuned weights and training data to public commons, highlighting practical impact for automated legislative analysis and transparent government AI.

Abstract

This report introduces LLaMandement, a state-of-the-art Large Language Model, fine-tuned by the French government and designed to enhance the efficiency and efficacy of processing parliamentary sessions (including the production of bench memoranda and documents required for interministerial meetings) by generating neutral summaries of legislative proposals. Addressing the administrative challenges of manually processing a growing volume of legislative amendments, LLaMandement stands as a significant legal technological milestone, providing a solution that exceeds the scalability of traditional human efforts while matching the robustness of a specialized legal drafter. We release all our fine-tuned models and training data to the community.
Paper Structure (25 sections, 9 figures)

This paper contains 25 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Generative Artificial Intelligence in the Service of the State
  3. Challenges and Dynamics of Amendment Procedures in French Parliamentary Practice
  4. The Practical Exercise of the Right of Amendment: An Administrative and AI Perspective
  5. Amendment attribution and similarity research: using simple language processing techniques
  6. Amendements summaries: using Large Language Models (LLMs)
  7. Related Work: Technical Solutions Through Large Language Models (LLMs)
  8. The Rise of Large Language Models in AI: A Paradigm Shift
  9. Beyond Language: Separating Substance from Political Messaging
  10. Challenges in Domain-Specific Applications
  11. Materials and Methods
  12. Model Selection
  13. Fine-Tuning: Low Rank Adaptation
  14. Data Collection
  15. Experiments
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Stacked bar chart displaying the aggregate amendments filed in the French Parliament from 2011 to 2022, segmented by originating chamber. The Senate's contributions are shown in red and the Assemblée Nationale's in blue. A red trend line highlights the overall direction of amendment activity throughout the last decade.
  • Figure 2: The image below shows an amendment in its most formal form as it should be submitted by any Member of the french Parliament. (1) At the very top, the document indicates the legislative bill to which the amendment is to be applied, providing context and reference for the proposed changes. (2) The amendment is identified either as targeting a specific existing article or as an 'Article additionnel' if it introduces new provisions not covered by existing articles. (3) The main body of the amendment follows, detailing the exact legislative changes proposed, known as the dispositif. This is where the parliamentarian's proposed text to be inserted into or to modify the bill is articulated. (4) The dispositif begins with a header that precisely locates where within the bill the amendment will take effect, such as specifying after which article the new content is to be inserted. (5) Finally, every amendment is accompanied by a rationale, outlined in the 'EXPOSÉ SOMMAIRE,' which provides the reasons and objectives behind the amendment, a mandatory element for the amendment's admissibility.
  • Figure 3: A bench memorandum detailing the government's position and a detailed response to an amendment proposal. The left side outlines the current situation, the proposed measure, budgetary impact, and a tax note. The right side provides the government's stance, offering an argument on the proposal's implications, followed by a detailed rationale for supporting or opposing the amendment.
  • Figure 4: Distribution of amendments and summaries across various 2023 bills in our dataset, showcasing the extensive legislative activity, reflecting LLaMandement's comprehensive fine-tuning scope.
  • Figure 5: Correlation of Model Size and Complexity to Performance in Legislative Text Analysis. The graph depicts the evolution of NLP models' performance scores as judged by legal drafters, with the size of each point correlating to the model's size.
  • ...and 4 more figures