Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA

TL;DR

This work presents Moxin-7B, a fully open-source large language model developed under the Model Openness Framework (MOF), and extends it with multimodal variants: Moxin-VLM for vision-language tasks, Moxin-VLA for vision-language-action in robotics, and Moxin-Chinese for enhanced Chinese capabilities. Built on open data and open frameworks (e.g., Prismatic VLM, OpenX-Embodiment, OpenVLA-OFT), the authors show that Moxin-VLM achieves competitive or superior performance against established backbones, while Moxin-VLA attains strong robotic-control performance under a streamlined fine-tuning recipe, and Moxin-Chinese demonstrates substantial gains on Chinese benchmarks after vocabulary expansion and translation-oriented fine-tuning. The evaluation spans open VLM benchmarks, robotics simulation, and cross-lingual tasks, evidencing the value of fully open approaches for reproducibility and practical impact. Overall, the paper argues that openness and transparent training data, methodologies, and release practices can propel open AI ecosystems without compromising performance. The release of models, data, and code aims to foster broader research collaboration and real-world deployment in vision-language and robotics domains.

Abstract

Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities. Leading this evolution are proprietary LLMs like GPT-4 and GPT-o1, which have captured widespread attention in the AI community due to their remarkable performance and versatility. Simultaneously, open-source LLMs, such as LLaMA and Mistral, have made great contributions to the ever-increasing popularity of LLMs due to the ease to customize and deploy the models across diverse applications. Moxin 7B is introduced as a fully open-source LLM developed in accordance with the Model Openness Framework, which moves beyond the simple sharing of model weights to embrace complete transparency in training, datasets, and implementation detail, thus fostering a more inclusive and collaborative research environment that can sustain a healthy open-source ecosystem. To further equip Moxin with various capabilities in different tasks, we develop three variants based on Moxin, including Moxin-VLM, Moxin-VLA, and Moxin-Chinese, which target the vision-language, vision-language-action, and Chinese capabilities, respectively. Experiments show that our models achieve superior performance in various evaluations. We adopt open-source framework and open data for the training. We release our models, along with the available data and code to derive these models.