HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

Abstract

We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.

Figures (13)

• Figure 1: Performance of HY-Embodied-0.5 MoT-2B on spatial and embodied benchmarks as well as downstream robot control tasks. HY-Embodied-0.5 pushes the frontier of embodied VLMs, while excelling in downstream real-world robot evaluations.
• Figure 2: HY-Embodied-0.5 Mixture-of-Transformers Architecture. The MoT design decouples the processing of visual and textual tokens by employing modality-specific QKV and FFN layers, alongside distinct attention mechanisms. Visual latent tokens and mixed optimization loss are employed to bridge and stress the relationships between modalities during large-scale training.
• Figure 3: Attention Computation of our Modality-Adaptive MoT. We visualize the attention computation under actual interleaved multi-modal sequences with distinct colors.
• Figure 4: Data Distribution for Pre-training and Mid-training Stages. We conduct large-scale embodied pre-training and mid-training to establish foundational and advanced physical-world competencies. The pre-training mixture encompasses over 200B tokens based on spatial, robotics, and visual perception tasks. The mid-training stage leverages over 12M high-quality QA pairs for complex real-world execution based on diverse spatial and embodied domains.
• Figure 5: Training Pipeline for HY-Embodied-0.5 Series. Large-scale pre-training establishes the models' foundational multi-modal representations and robust spatial-embodied perception. The subsequent Embodied Post-training phase explicitly enhances complex reasoning capabilities through iterative self-evolution and reinforcement learning. Finally, we employ on-policy distillation to effectively transfer the knowledge from large variants to edge deployment.