AGDC: Autoregressive Generation of Variable-Length Sequences with Joint Discrete and Continuous Spaces

TL;DR

AGDC addresses the core limitation of discretization-based sequence generation by jointly modeling discrete tokens and continuous values in variable-length sequences. It introduces an EOS logit adjustment mechanism and a differentiable length regularization term, enabling context-aware termination and controllable sequence length. The framework uses atomic units with parallel discrete and diffusion-based continuous branches, validated on ContLayNet, graphic layouts, and text-to-SVG tasks, where it outperforms fixed-schema and discretization baselines. ContLayNet provides a high-precision, design-rule-focused benchmark to rigorously assess functional correctness in real-world hybrid vector data, underscoring AGDC's practical impact for high-precision engineering and vector graphics generation.

Abstract

Transformer-based autoregressive models excel in data generation but are inherently constrained by their reliance on discretized tokens, which limits their ability to represent continuous values with high precision. We analyze the scalability limitations of existing discretization-based approaches for generating hybrid discrete-continuous sequences, particularly in high-precision domains such as semiconductor circuit designs, where precision loss can lead to functional failure. To address the challenge, we propose AGDC, a novel unified framework that jointly models discrete and continuous values for variable-length sequences. AGDC employs a hybrid approach that combines categorical prediction for discrete values with diffusion-based modeling for continuous values, incorporating two key technical components: an end-of-sequence (EOS) logit adjustment mechanism that uses an MLP to dynamically adjust EOS token logits based on sequence context, and a length regularization term integrated into the loss function. Additionally, we present ContLayNet, a large-scale benchmark comprising 334K high-precision semiconductor layout samples with specialized evaluation metrics that capture functional correctness where precision errors significantly impact performance. Experiments on semiconductor layouts (ContLayNet), graphic layouts, and SVGs demonstrate AGDC's superior performance in generating high-fidelity hybrid vector representations compared to discretization-based and fixed-schema baselines, achieving scalable high-precision generation across diverse domains.

Figures (13)

• Figure 1: Impact of discretization. Discretization fundamentally compromises the scalability of precision. For precision analysis, refer to \ref{['subsec:precision']}.
• Figure 2: Atomic unit representation.
• Figure 3: Overview of AGDC.
• Figure 4: The ContLayNet dataset. Left: Example visualization of a sample. Right: Comparison showing functional failure when ContLayNet samples get discretized to low precision.
• Figure 5: Qualitative results on ContLayNet in completion task given 50 / 100 layers. AGDC illustrate clearly superior performance to LT and DLT.