Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications

TL;DR

Yuan3.0 Flash tackles enterprise-scale multimodal reasoning by combining a 40B MoE language backbone with a lightweight visual alignment module and a Reflection-aware RAPO training regime. The approach mitigates overthinking through RIRM guided rewards, adaptive sampling, and a unified reward framework that blends verifiable and generative scoring. Empirical results demonstrate strong enterprise performance across retrieval, table understanding, summarization, and tool usage, while achieving notable token efficiency compared with larger models. The work provides an open-source, end-to-end framework including data curation, multitask RL, and evaluation pipelines to accelerate deployment and research in enterprise AI systems.

Abstract

We introduce Yuan3.0 Flash, an open-source Mixture-of-Experts (MoE) MultiModal Large Language Model featuring 3.7B activated parameters and 40B total parameters, specifically designed to enhance performance on enterprise-oriented tasks while maintaining competitive capabilities on general-purpose tasks. To address the overthinking phenomenon commonly observed in Large Reasoning Models (LRMs), we propose Reflection-aware Adaptive Policy Optimization (RAPO), a novel RL training algorithm that effectively regulates overthinking behaviors. In enterprise-oriented tasks such as retrieval-augmented generation (RAG), complex table understanding, and summarization, Yuan3.0 Flash consistently achieves superior performance. Moreover, it also demonstrates strong reasoning capabilities in domains such as mathematics, science, etc., attaining accuracy comparable to frontier model while requiring only approximately 1/4 to 1/2 of the average tokens. Yuan3.0 Flash has been fully open-sourced to facilitate further research and real-world deployment: https://github.com/Yuan-lab-LLM/Yuan3.0.

Figures (9)

• Figure 2: Overall architecture of Yuan3.0 Flash and MoE-based Language Backbone. The left figure depicts the proposed Yuan3.0 architecture, which comprises three integral components: (1) a ViT Encoder responsible for processing and encoding input images; (2) a lightweight MLP projector with SwiGLU activations to align visual features with the textual token space; and (3) a MoE-based Language Model that serves as the Language Decoder. The right figure presents the language backbone with Localizing Filtering-based Attention (LFA)
• Figure 3: (a) An example of Deepseek-R1's reasoning process, showing repetitive "reflection" behavior after deriving the "first answer". (b) Average token consumption breakdown of DeepseekR1-Distill-1.5B and Deepseek-R1 across AIME 2024 and MATH-500 benchmarks. The light-colored segments indicate token consumption before the "first answer" appears, while the dark-colored segments correspond to token consumption during the "reflection" phase.
• Figure 4: Illustration of Reflection Inhibition Reward Mechanism.
• Figure 5: (a) Example of DeepseekR1-Distill-1.5B’s reasoning process after trained with RIRM. (b) Comparison of average token consumption of DeepseekR1-Distill-1.5B before and after trained with RIRM in AIME 2024 and MATH-500 benchmarks. The light-colored segments indicate token consumption before the "first answer" appears, while the dark-colored segments correspond to token consumption during the "reflection" phase.
• Figure 6: Training and testing accuracy of DeepSeek-R1-Distill-Qwen-1.5B under DAPO with and without ADS.