Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Aligning Language Models for Icelandic Legal Text Summarization

Þórir Hrafn Harðarson, Hrafn Loftsson, Stefán Ólafsson

TL;DR

This work investigates aligning Icelandic legal text summarization with domain-specific language standards using preference-based training. By combining domain-focused pretraining, instruction fine-tuning, and methods such as Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF), the study shows that preference training can improve legal accuracy but does not consistently enhance general Icelandic language quality. The results also reveal a disconnect between automated metrics (e.g., ROUGE) and human expert judgments, underscoring the need for qualitative evaluation in legal NLP. The findings highlight the importance of language-specific pre-training and suggest future work in retrieval-augmented generation and larger, more diverse datasets to achieve practically reliable Icelandic legal text summarization.

Abstract

The integration of language models in the legal domain holds considerable promise for streamlining processes and improving efficiency in managing extensive workloads. However, the specialized terminology, nuanced language, and formal style of legal texts can present substantial challenges. This study examines whether preference-based training techniques, specifically Reinforcement Learning from Human Feedback and Direct Preference Optimization, can enhance models' performance in generating Icelandic legal summaries that align with domain-specific language standards and user preferences. We compare models fine-tuned with preference training to those using conventional supervised learning. Results indicate that preference training improves the legal accuracy of generated summaries over standard fine-tuning but does not significantly enhance the overall quality of Icelandic language usage. Discrepancies between automated metrics and human evaluations further underscore the importance of qualitative assessment in developing language models for the legal domain.

Aligning Language Models for Icelandic Legal Text Summarization

TL;DR

This work investigates aligning Icelandic legal text summarization with domain-specific language standards using preference-based training. By combining domain-focused pretraining, instruction fine-tuning, and methods such as Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF), the study shows that preference training can improve legal accuracy but does not consistently enhance general Icelandic language quality. The results also reveal a disconnect between automated metrics (e.g., ROUGE) and human expert judgments, underscoring the need for qualitative evaluation in legal NLP. The findings highlight the importance of language-specific pre-training and suggest future work in retrieval-augmented generation and larger, more diverse datasets to achieve practically reliable Icelandic legal text summarization.

Abstract

The integration of language models in the legal domain holds considerable promise for streamlining processes and improving efficiency in managing extensive workloads. However, the specialized terminology, nuanced language, and formal style of legal texts can present substantial challenges. This study examines whether preference-based training techniques, specifically Reinforcement Learning from Human Feedback and Direct Preference Optimization, can enhance models' performance in generating Icelandic legal summaries that align with domain-specific language standards and user preferences. We compare models fine-tuned with preference training to those using conventional supervised learning. Results indicate that preference training improves the legal accuracy of generated summaries over standard fine-tuning but does not significantly enhance the overall quality of Icelandic language usage. Discrepancies between automated metrics and human evaluations further underscore the importance of qualitative assessment in developing language models for the legal domain.

Paper Structure

This paper contains 21 sections, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. RQ:
  3. Background and Related Work
  4. Methods
  5. Datasets
  6. Domain Specific Further Pre-Training
  7. Instruction Fine-Tuning
  8. Preference Training
  9. Human Evaluation
  10. Results
  11. Instruction Fine-Tuning
  12. Preference Training
  13. Direct Preference Optimization
  14. Reinforcement Learning from Human Feedback
  15. Human Evaluation
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Mean reward and KL-divergence for GPT-SW3 1.3B after 20 epochs of training using the PPO reinforcement learning algorithm.