Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction

TL;DR

Ming-Lite-Uni tackles unified multimodal interaction by pairing a frozen MLLM with a trainable diffusion generator within a native autoregressive framework. It introduces multi-scale learnable tokens and a scale-wise representation alignment mechanism to bridge visual and semantic spaces, enabling both text-to-image generation and instruction-based editing. Trained on extensive image-text and image-editing data, it achieves strong multimodal understanding and competitive generation performance, while remaining open-source in an alpha release. The work aims to accelerate progress toward unified multimodal models and broader AI capabilities by providing accessible code and weights for community experimentation.

Abstract

We introduce Ming-Lite-Uni, an open-source multimodal framework featuring a newly designed unified visual generator and a native multimodal autoregressive model tailored for unifying vision and language. Specifically, this project provides an open-source implementation of the integrated MetaQueries and M2-omni framework, while introducing the novel multi-scale learnable tokens and multi-scale representation alignment strategy. By leveraging a fixed MLLM and a learnable diffusion model, Ming-Lite-Uni enables native multimodal AR models to perform both text-to-image generation and instruction based image editing tasks, expanding their capabilities beyond pure visual understanding. Our experimental results demonstrate the strong performance of Ming-Lite-Uni and illustrate the impressive fluid nature of its interactive process. All code and model weights are open-sourced to foster further exploration within the community. Notably, this work aligns with concurrent multimodal AI milestones - such as ChatGPT-4o with native image generation updated in March 25, 2025 - underscoring the broader significance of unified models like Ming-Lite-Uni on the path toward AGI. Ming-Lite-Uni is in alpha stage and will soon be further refined.

Figures (7)

• Figure 1: The output results and multimodal interactive demos of Ming-Lite-Uni. Our model supports basic multimodal chatting, text-to-image generation, image editing, and image style transfer based on textual instructions.
• Figure 2: The framework of Ming-Lite-Uni. Our model fixes the MLLM and fine-tunes the diffusion model through the newly designed multi-scale learnable tokens, multi-scale representation alignment, and connector.
• Figure 3: The AR part of Ming-Lite-Uni. Our model reuses the M2-omni MLLM as a frozen token prediction module, retaining only its text and image branches. The pretraining procedure and dataset of the AR model are consistent with our previous work, please refer to guo2025m2 for details.
• Figure 4: Basic image-text training pairs of Ming-Lite-Uni.
• Figure 5: Ming-Lite-Uni image editing data examples in the training set.