Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection

TL;DR

The paper systematically compares full fine-tuning, LoRA-based parameter-efficient tuning, and zero-shot prompting for detecting unfair clauses in Terms of Service documents. Using CLAUDETTE-ToS and a large multilingual ToS corpus, it shows that full fine-tuning provides the strongest precision-recall balance, while LoRA achieves high recall with substantially reduced memory requirements, and zero-shot prompting offers rapid deployment with lower precision. The study delivers practical baselines and design guidance for domain-adapted legal NLP, highlighting trade-offs between accuracy, resource use, and scalability for real-world compliance auditing. Its deployment on noisy web-scale data demonstrates the feasibility of lightweight yet effective unfair clause detectors for large-scale regulatory monitoring.

Abstract

Large Language Models (LLMs) have transformed text understanding, yet their adaptation to specialized legal domains remains constrained by the cost of full fine-tuning. This study provides a systematic evaluation of fine tuning, parameter efficient adaptation (LoRA, QLoRA), and zero-shot prompting strategies for unfair clause detection in Terms of Service (ToS) documents, a key application in legal NLP. We finetune BERT and DistilBERT, apply 4-bit Low-Rank Adaptation (LoRA) to models such as TinyLlama, LLaMA 3B/7B, and SaulLM, and evaluate GPT-4o and O-versions in zero-shot settings. Experiments on the CLAUDETTE-ToS benchmark and the Multilingual Scraper Corpus show that full fine-tuning achieves the strongest precision recall balance, while LoRA-based models provide competitive recall with up to 3x lower memory cost. These findings highlight practical design trade-offs for efficient and domain-adapted LLMs, contributing open baselines for fine-tuning research in legal text processing.

Figures (1)

• Figure 1: System Prompt for zero-shot prompting