Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Edits

Neha Kalibhat, Zi Wang, Prasoon Bajpai, Drew Proud, Wenjun Zeng, Been Kim, Mani Malek

TL;DR

This work tackles the challenge of interpreting and controlling black-box generative models by learning a natural language constitution that maps atomic concept edits (ACEs) to predictable behavioral changes. The authors formalize ACEs as add, remove, or replace operations on prompt concepts and optimize constitutions via a surrogate LLM and an evolutionary process to guide ACE generation. They demonstrate the approach on Word Count, Math, and text-to-image alignment tasks, using diverse models and autoraters, and show that constitution-guided ACE achieves about 1.86x higher success rates than non-constitution baselines while preserving prompt diversity. The framework yields actionable, generalizable insights into how prompting concepts shape model outputs and enables more efficient, interpretable steering across modalities, with potential extensions to broader inputs and operator spaces.

Abstract

We introduce a black-box interpretability framework that learns a verifiable constitution: a natural language summary of how changes to a prompt affect a model's specific behavior, such as its alignment, correctness, or adherence to constraints. Our method leverages atomic concept edits (ACEs), which are targeted operations that add, remove, or replace an interpretable concept in the input prompt. By systematically applying ACEs and observing the resulting effects on model behavior across various tasks, our framework learns a causal mapping from edits to predictable outcomes. This learned constitution provides deep, generalizable insights into the model. Empirically, we validate our approach across diverse tasks, including mathematical reasoning and text-to-image alignment, for controlling and understanding model behavior. We found that for text-to-image generation, GPT-Image tends to focus on grammatical adherence, while Imagen 4 prioritizes atmospheric coherence. In mathematical reasoning, distractor variables confuse GPT-5 but leave Gemini 2.5 models and o4-mini largely unaffected. Moreover, our results show that the learned constitutions are highly effective for controlling model behavior, achieving an average of 1.86 times boost in success rate over methods that do not use constitutions.

Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Edits

TL;DR

This work tackles the challenge of interpreting and controlling black-box generative models by learning a natural language constitution that maps atomic concept edits (ACEs) to predictable behavioral changes. The authors formalize ACEs as add, remove, or replace operations on prompt concepts and optimize constitutions via a surrogate LLM and an evolutionary process to guide ACE generation. They demonstrate the approach on Word Count, Math, and text-to-image alignment tasks, using diverse models and autoraters, and show that constitution-guided ACE achieves about 1.86x higher success rates than non-constitution baselines while preserving prompt diversity. The framework yields actionable, generalizable insights into how prompting concepts shape model outputs and enables more efficient, interpretable steering across modalities, with potential extensions to broader inputs and operator spaces.

Abstract

We introduce a black-box interpretability framework that learns a verifiable constitution: a natural language summary of how changes to a prompt affect a model's specific behavior, such as its alignment, correctness, or adherence to constraints. Our method leverages atomic concept edits (ACEs), which are targeted operations that add, remove, or replace an interpretable concept in the input prompt. By systematically applying ACEs and observing the resulting effects on model behavior across various tasks, our framework learns a causal mapping from edits to predictable outcomes. This learned constitution provides deep, generalizable insights into the model. Empirically, we validate our approach across diverse tasks, including mathematical reasoning and text-to-image alignment, for controlling and understanding model behavior. We found that for text-to-image generation, GPT-Image tends to focus on grammatical adherence, while Imagen 4 prioritizes atmospheric coherence. In mathematical reasoning, distractor variables confuse GPT-5 but leave Gemini 2.5 models and o4-mini largely unaffected. Moreover, our results show that the learned constitutions are highly effective for controlling model behavior, achieving an average of 1.86 times boost in success rate over methods that do not use constitutions.
Paper Structure (26 sections, 2 equations, 16 figures, 3 tables, 1 algorithm)

This paper contains 26 sections, 2 equations, 16 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Interpretability and causality.
  4. Adversarial attacks.
  5. Active learning and testing.
  6. Prompt optimization.
  7. "Constitutions" in AI.
  8. Concepts.
  9. Problem Formulation
  10. Atomic Concept Edits (ACEs)
  11. Goal: Learning Constitutions for ACEs
  12. Our Methods
  13. Constitution-guided ACE Module
  14. Constitution Optimization Module
  15. Using Constitutions for Model Behavior Control
  16. ...and 11 more sections

Figures (16)

  • Figure 1: Qualitative Examples of ACE: We illustrate how ACE may create mutations that eventually satisfy a given task. In the top-left panel, we demonstrate 'Word Count' prompts on Gemini-2.5-Flash with the goal of adherence to a word constraint. In the top-right panel, we demonstrate the 'Math' prompts on GPT o4-mini with the goal of making the problem difficult to solve. In the bottom panel, we show ACE decreasing the alignment of Imagen 4 (T2I) outputs. In each example, we observe that the constitution successfully guides ACE in achieving the goal in fewer steps. More examples shown in Figure \ref{['fig:ace_examples']}.
  • Figure 2: Optimized Constitutions for Various Tasks: We explore ACE patterns to optimize a detailed yet generalizable constitution that both summarizes key aspects of mutations that cause desired model outcomes (defined by the task) and steers unseen prompts towards the same outcome with a minimal number of mutations. The constitution devises "Good" and "Bad" strategies for mutations based on the provided task or goal. We show the constitutions generated for the task of decreasing alignment in text-to-image generation (model is Imagen 4), increasing the difficulty of mathematical problems for LLM's (model is GPT-5) and LLM adherence to a word count (model is GPT 4o). More detailed constitutions are shown in the Appendix.
  • Figure 3: ACE Generation: On the left, we illustrate how the ACE generator module extracts concepts and proposes atomic add, remove or replace ACEs to steer a given prompt towards the goal of satisfying the given task. ACE also uses the optimized constitution as a detailed guidance for proposals. Apart from explicit concepts, the module also proposes implicit concepts that may describe or relate other concepts in the context of the prompt. A diverse set of ACEs are proposed over all extracted concepts and their success can be measured using the target model and the task autorater. ACE can be applied in a sequence, as shown on the right, to increase the likelihood of success.
  • Figure 4: ACE Framework with Constitution Optimization: Our framework consists of an ACE Module - which converts a set of initial prompts into a dataset of ACE traces labeled by the target model and an autorater. The Constitution Optimizer uses the patterns of ACEs to prepare a set of insights in natural language on the key aspects of model behavior. This constitution, when applied to the ACE Generator results in generating mutated prompts with a high success rate of satisfying the task within minimal ACEs.
  • Figure 5: Probability of ACEs Succeeding: We measure the probability of ACE satisfying the goal task within the given number of steps (or length of ACE sequences), represented in the x-axes. We observe that using an optimized constitution with ACE helps achieve the goal faster (blue curves). Within 4 steps, we observe that ACE achieves a high success rate ($>$ 0.8%) across all models and tasks.
  • ...and 11 more figures

Theorems & Definitions (1)

  • Definition 3.1