GenCtrl -- A Formal Controllability Toolkit for Generative Models

TL;DR

GenCtrl presents a formal control-theoretic framework for gauging the controllability of dialogue-based generative systems, defining $\mathcal{R}_t$ and $\mathcal{C}_t^{\alpha}$ for output reachability and partial controllability with PAC guarantees. It introduces Monte Carlo methods with a $\gamma$-quantized, coarse-grained approach to overcome the discrete bottleneck inherent in prompts and token-based generations, providing bounds that scale with sample size $m$ and discretization. The authors validate the framework on LLMs and T2IMs, revealing that controllability is not guaranteed and is highly sensitive to task, model size, prompting, and input distributions, while delivering an open-source toolkit for rigorous analysis. The work formalizes fundamental limits of controllable generation, enabling principled comparisons of prompting strategies and guiding safety-aware design for future AI systems.

Abstract

As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits.

Figures (17)

• Figure 1: Dialogue Process. (Left) Schema of a dialogue process as a control process, showing the roles of each of the concepts introduced in \ref{['sec:dialogue_process']}. \ref{['mythm:abstract', 'mythm:controllability']} tell how many inputs and initial states (respectively) should be sampled to answer Q1 and Q2 with confidence $\delta$. (Right) Example of dialogue process.
• Figure 2: (Top, Middle) 5-turn Dialogue Process trajectories for formality task. Controllable set dynamics are shown for (left to right) models SmolLM3-3B, Qwen3-4B, and Gemma3-4B on a text formality control task, using 0-shot (top) and 5-shot (bottom) prompting as the initial input. Each linecolor represents a different initial state. The $\alpha$-controllable sets ($\alpha=0.1$) are shown in yellow, where full controllability (best-case) would be seen by an entirely yellow $t=5$. While none of the models are fully controllable 0-shot, and all show a formal bias, Gemma3-4B and Qwen3-4B are the most controllable with $5$ shots by $t=5$ (confidence $\delta=0.05$). (Bottom) Summarized metrics for 5-shot at $t=5$. The left figure shows the final output in the DP as a function of the requested input. The next figures show violin plots of each metric on the formality task, where each point is a metric for a single $x_0$, demonstrating Qwen3-4B is the most controllable and faithful to the user request for this setting ($\mathrm{cvg}=1.0$, median $\mathrm{MAE}=0.09$).
• Figure 3: Larger models are more controllable and calibrated on text formality. For Qwen sizes ranging from 0.6B to 14B (x-axis), we requested text formalities ranging in $[0,1]$, with 0-shot prompting and one dialogue turn. While controllability (right) increases reliably up to 14B, the correlation (left plots) between the user request and the output formality, given by $R$, plateaus around 8B. All calibration metrics ($R$, $\rho$, MAE) increase most drastically for smaller sizes (0.6B $\to$1.7B) and appear to saturate for larger sizes.
• Figure 4: Object generation task for T2IMs. We prompt the model with "White background. [N] [obj]s." with $\texttt{N}=\{0\ldots 20\}$ and $\texttt{obj}=\{\text{80 COCO classes}\}$. The left figure shows the average output object count as a function of the requested input. The next figures contain violin plots of each evaluation metric $\forall\texttt{obj}$, evidencing differences in models. Notably, FLUX-s achieves a median $\rho, R > 0.9$ and a median $\mathrm{MAE}=3.52$, showing a much better controllability and faithfulness than the rest.
• Figure A.5: Requesting string length in $\{1\cdots 10\}$ with 5-shot prompting. We ask the LLM to generate a string of length $N$ characters, where $N \sim \text{Unif}\{1\cdots 10\}$. The controllable set estimates are shown on the (right), with Gemma3-4B displaying the highest $\alpha$-controllability ($\alpha=0.1$), to 80% of the desired range. The distribution of string lengths $y_T$ is plotted with respect to the initial request on the (left). In general, especially for Qwen3-4B and Gemma3-4B, the generations are faithful to the request no matter the initial prompt, seen by points landing on the line $y=x$ (dashed). This is corroborated by the (middle) three plots, however, SmolLM3-3B shows high variance of the faithfulness metrics $\rho$, $R$, and MAE across initial states.

Theorems & Definitions (24)

• Definition 1
• Definition 2
• Definition 3
• Definition 4: $\gamma$-quantized Reachable Set
• Definition 5: $p$-Approximate Measurement Value
• Definition 6: $\alpha$-controllable set
• Definition 7
• Definition 8: $\epsilon$-accurate forward reachable set, devonport_data-driven_2019
• Theorem 9: Output forward reachability, adapted devonport_data-driven_2019
• Lemma 10: Deterministic intervened LLMs' reachable sets are countable