Kelix Technique Report

TL;DR

Kelix presents a fully discrete, autoregressive unified model for multimodal understanding and generation by introducing multi-token visual tokenization and a Next-Block Prediction training regime. The architecture couples a discrete image tokenizer (Kelix-Tok), a unified LLM backbone, and a diffusion-based detokenizer to achieve competitive performance with continuous-feature models while remaining fully discrete. Key contributions include expanding visual coding capacity with independent sub-codebooks, a block-wise autoregressive scheme that keeps sequence length manageable, and a comprehensive four-stage LLM training protocol plus a two-stage diffusion-based generation path. Extensive experiments show state-of-the-art results among comparable-scale unified models on both understanding and generation benchmarks, highlighting strong cross-modal alignment and robust instruction-following capabilities. The work advances unified multimodal intelligence and suggests promising directions for extending to video and beyond.

Abstract

Autoregressive large language models (LLMs) scale well by expressing diverse tasks as sequences of discrete natural-language tokens and training with next-token prediction, which unifies comprehension and generation under self-supervision. Extending this paradigm to multimodal data requires a shared, discrete representation across modalities. However, most vision-language models (VLMs) still rely on a hybrid interface: discrete text tokens paired with continuous Vision Transformer (ViT) features. Because supervision is largely text-driven, these models are often biased toward understanding and cannot fully leverage large-scale self-supervised learning on non-text data. Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling, showing promising progress toward unified understanding and generation. Yet existing discrete vision tokens frequently lose information due to limited code capacity, resulting in noticeably weaker understanding than continuous-feature VLMs. We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.

Figures (9)

• Figure 1: Benchmark performance of Kelix. Compared with prior discrete unified models, Kelix significantly improves understanding, even matching continuous understanding-only VLMs, while achieving competitive image generation results with current SOTA.
• Figure 2: The auto-regressive inference workflow of our method, including the Kelix-Tok, and unified semantic understanding and image generation.
• Figure 3: The auto-regressive training workflow of our method, including the Kelix Tokenizer, and Unified LLM model and Image DiT.
• Figure 5: Semantic tokenizer training recipe. In Stage 1, we freeze the ViT and LLM backbones. To ensure that the commitment loss can be optimized, we insert the MLP and SimVQ up-projector before VQ codebook. In Stage2 and Stage3, we enable the gradients of ViT and Qwen3.
• Figure 6: The LLM training recipe of Kelix, we first pre-train the Block Decoder and the image token embedding for a better warm start, and then unifying the image token and text token to a same embedding table for full-parameter tuning.