IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking

TL;DR

IterGen addresses semantic correctness and privacy concerns in LLM generation by introducing grammar-symbol-based iterative generation with backtracking. It employs an LR-parser-backed symbol-position map and a KV cache to enable forward and backward navigation within a CFG-defined output, enabling targeted semantic corrections. Across SQL, privacy leakage, and Vega-Lite tasks, it achieves significant gains over baseline methods, including an 18.5% absolute improvement in SQL accuracy and complete privacy leakage mitigation, while maintaining efficiency via cached state. The approach provides a practical open-source tool for enforcing complex semantic constraints in structured outputs and paves the way for broader applications in code, data visualization, and natural language generation.

Abstract

Large Language Models (LLMs) are widely used for tasks such as natural language and code generation, but their outputs often suffer from issues like hallucination, toxicity, and incorrect results. Current libraries for structured LLM generation rely on left-to-right decoding without support for backtracking, limiting the ability to correct or refine outputs mid-generation. To address this, we introduce IterGen, a user-friendly library for iterative, grammar-guided LLM generation that enables users to move both forward and backward within the generated output based on grammar symbols. By leveraging a symbol-to-position mapping and maintaining the key-value (KV) cache state, IterGen ensures efficient and structured generation while allowing for corrections during the process. We demonstrate IterGen's effectiveness in two important applications: reducing privacy leakage in LLM outputs and improving the accuracy of LLM-generated SQL and Vega-Lite queries. Our code and additional resources are available at https://structuredllm.com.

Figures (6)

• Figure 1: In our workflow, a user program utilizing the IterGen manages LLM generation through forward and backward calls. For each prompt $O_0$, IterGen maintains a session that includes a decoding trace, a symbol position map, and a key-value (KV) cache. Using the LR parser IterGen incrementally parses partially generated output $O_k$ and continuously updates the symbol position map to track the locations of symbols from the grammar in $O_k$.
• Figure 2: On every reduce operation the IterGen updates the position of the reduced symbol in the symbol position map.
• Figure 3: Code using IterGen for LLM-based SQL Generation
• Figure 4: Scatter plot showing the total number of iterations (x-axis) and the number of backward calls (y-axis) for the tasks in the Vega-lite case study on Llama-3.2B. The red dotted line represents the maximum iteration limit of 50. The size of each scatter point is scaled logarithmically to show the frequency of tasks with that specific coordinate.
• Figure 5: Code using IterGen for reducing Privacy Leakage of email addresses through LLMs