Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law

Ilija Lichkovski, Alexander Müller, Mariam Ibrahim, Tiwai Mhundwa

TL;DR

This paper introduces EU-Agent-Bench, a verifiable benchmark designed to measure whether LLM agents stay within EU law when handling benign prompts that could lead to illegal actions. It uses a rubric tied to EU regulations to evaluate single-turn tool calls across six legal domains, with a data-augmented, human-curated dataset of 600 prompts. Across seven LLM checkpoints, compliance varies widely (best around 55%, worst around 38%), and model scale does not reliably improve legality. The work provides a public preview data set, a private hold-out, and an evaluation protocol to drive safer, auditable agent behavior, and discusses limitations and future directions toward multi-turn, causal, and cross-jurisdictional benchmarks.

Abstract

Large language models (LLMs) are increasingly deployed as agents in various contexts by providing tools at their disposal. However, LLM agents can exhibit unpredictable behaviors, including taking undesirable and/or unsafe actions. In order to measure the latent propensity of LLM agents for taking illegal actions under an EU legislative context, we introduce EU-Agent-Bench, a verifiable human-curated benchmark that evaluates an agent's alignment with EU legal norms in situations where benign user inputs could lead to unlawful actions. Our benchmark spans scenarios across several categories, including data protection, bias/discrimination, and scientific integrity, with each user request allowing for both compliant and non-compliant execution of the requested actions. Comparing the model's function calls against a rubric exhaustively supported by citations of the relevant legislature, we evaluate the legal compliance of frontier LLMs, and furthermore investigate the compliance effect of providing the relevant legislative excerpts in the agent's system prompt along with explicit instructions to comply. We release a public preview set for the research community, while holding out a private test set to prevent data contamination in evaluating upcoming models. We encourage future work extending agentic safety benchmarks to different legal jurisdictions and to multi-turn and multilingual interactions. We release our code on \href{https://github.com/ilijalichkovski/eu-agent-bench}{this URL}.

EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law

TL;DR

This paper introduces EU-Agent-Bench, a verifiable benchmark designed to measure whether LLM agents stay within EU law when handling benign prompts that could lead to illegal actions. It uses a rubric tied to EU regulations to evaluate single-turn tool calls across six legal domains, with a data-augmented, human-curated dataset of 600 prompts. Across seven LLM checkpoints, compliance varies widely (best around 55%, worst around 38%), and model scale does not reliably improve legality. The work provides a public preview data set, a private hold-out, and an evaluation protocol to drive safer, auditable agent behavior, and discusses limitations and future directions toward multi-turn, causal, and cross-jurisdictional benchmarks.

Abstract

Large language models (LLMs) are increasingly deployed as agents in various contexts by providing tools at their disposal. However, LLM agents can exhibit unpredictable behaviors, including taking undesirable and/or unsafe actions. In order to measure the latent propensity of LLM agents for taking illegal actions under an EU legislative context, we introduce EU-Agent-Bench, a verifiable human-curated benchmark that evaluates an agent's alignment with EU legal norms in situations where benign user inputs could lead to unlawful actions. Our benchmark spans scenarios across several categories, including data protection, bias/discrimination, and scientific integrity, with each user request allowing for both compliant and non-compliant execution of the requested actions. Comparing the model's function calls against a rubric exhaustively supported by citations of the relevant legislature, we evaluate the legal compliance of frontier LLMs, and furthermore investigate the compliance effect of providing the relevant legislative excerpts in the agent's system prompt along with explicit instructions to comply. We release a public preview set for the research community, while holding out a private test set to prevent data contamination in evaluating upcoming models. We encourage future work extending agentic safety benchmarks to different legal jurisdictions and to multi-turn and multilingual interactions. We release our code on \href{https://github.com/ilijalichkovski/eu-agent-bench}{this URL}.
Paper Structure (27 sections, 2 equations, 1 figure, 2 tables)

This paper contains 27 sections, 2 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. EU-Agent-Bench
  3. Evaluation Setup
  4. Evaluation Results
  5. Case Study: Explicit Inclusion of EU Legislation
  6. Discussion and Conclusion
  7. Limitations
  8. Conclusion
  9. Related Work
  10. Agentic safety
  11. LLMs in a legal context
  12. Legal Scenarios
  13. Data Protection
  14. Scientific Misconduct
  15. Copyright
  16. ...and 12 more sections

Figures (1)

  • Figure 1: Bar plot showing difference in the percentage of legal actions taken when Gemini 2.5 Flash had access to the content of the relevant EU legislation (injected regulations) in the system prompt versus when it did not (normal).