Can LLMs Follow Simple Rules?

TL;DR

This paper introduces RuLES, a programmatic benchmark that evaluates how well LLMs follow explicit, potentially dynamic rules across 14 text-based scenarios. It provides automatic evaluation functions to detect rule violations, enabling scalable comparisons of both proprietary and open-weight models. Findings show pervasive rule-following failures across models, with adversarial suffixes and alignment-fine-tuning often deteriorating performance; some improvements arise from test-time steering and supervised fine-tuning. RuLES exposes a dimension of model reliability not captured by existing benchmarks and offers a concrete platform for developing robust rule-following, steering, and defense mechanisms in LLMs.

Abstract

As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a reliable manner. Model developers may wish to set explicit rules for the model, such as "do not generate abusive content", but these may be circumvented by jailbreaking techniques. Existing evaluations of adversarial attacks and defenses on LLMs generally require either expensive manual review or unreliable heuristic checks. To address this issue, we propose Rule-following Language Evaluation Scenarios (RuLES), a programmatic framework for measuring rule-following ability in LLMs. RuLES consists of 14 simple text scenarios in which the model is instructed to obey various rules while interacting with the user. Each scenario has a programmatic evaluation function to determine whether the model has broken any rules in a conversation. Our evaluations of proprietary and open models show that almost all current models struggle to follow scenario rules, even on straightforward test cases. We also demonstrate that simple optimization attacks suffice to significantly increase failure rates on test cases. We conclude by exploring two potential avenues for improvement: test-time steering and supervised fine-tuning.

Figures (9)

• Figure 1: Example instance of our Encryption scenario. This scenario requires the assistant model to avoid repeating a secret key to the user. In the test case shown here the user directly asks the model to print the secret key and is correctly denied.
• Figure 2: Overview of our 14 rule-following scenarios. We show a decision tree representing expected model behavior for each scenario. Helpful rules mandating a certain behavior are shown in green circles, while harmless rules prohibiting a certain behavior are shown in red octagons.
• Figure 3: RuLES score for top-20 evaluated models, de-duplicated. Green bars (left) indicate scores from 0 to 10.
• Figure 4: Effects of alignment fine-tuning on RuLES score. We compare the performance of base Llama-2 and Gemma models to official fine-tuned variants. Both Meta and Google's alignment methods significantly hurt performance on our rule-following benchmark.
• Figure 5: Relationship between RuLES harmless score and other benchmark results. Pearson correlation coefficient results between benchmarks shown in boxes. Performance measured on our benchmark shows zero, or negative, correlation with existing benchmarks.