Linear Feedback Control Systems for Iterative Prompt Optimization in Large Language Models
Rupesh Raj Karn
TL;DR
This work reframes iterative prompt optimization for large language models as a linear feedback control problem, introducing a PID-based loop where the control signal $u(t)$ updates the prompt $p(t)$ to steer the LLM output toward a setpoint $r(t)$. The authors formalize the closed-loop structure, account for LLM nonlinearity and stochasticity with noise terms $\eta(t)$ and $\nu(t)$, and detail how the PID components influence tokenization, embedding, positional encoding, and transformer-based generation. A thorough impact analysis shows that the proportional term $K_p e(t)$ drives immediate effects while the integral and derivative terms contribute long-term stability and smoothing, with session-based memory further enhancing performance. The paper also compares PID with Lead-Lag, LQR, and Fuzzy Controllers and presents an FPGA design use-case that demonstrates iterative prompt refinement guiding resource utilization toward a setpoint, illustrating practical applicability and potential integration into real-time systems.
Abstract
Large Language Models (LLMs) have revolutionized various applications by generating outputs based on given prompts. However, achieving the desired output requires iterative prompt refinement. This paper presents a novel approach that draws parallels between the iterative prompt optimization process in LLMs and feedback control systems. We iteratively refine the prompt by treating the deviation between the LLM output and the desired result as an error term until the output criteria are met. This process is akin to a feedback control system, where the LLM, despite being non-linear and non-deterministic, is managed using principles from linear feedback control systems. We explore the application of different types of controllers within this framework, providing a mathematical foundation for integrating linear feedback control mechanisms with LLMs.