AuroraEdge-V-2B: A Faster And Stronger Edge Visual Large Language Model

TL;DR

AuroraEdge-V-2B targets practical industrial deployment by delivering a compact 2B-parameter VLLM with a novel token compressor and fusion module. By compressing visual tokens from 256 to 64 and injecting compensated visual information into text tokens, the model achieves high accuracy while substantially reducing FLOPs and latency. A three-stage training pipeline and diverse synthesized data underpin strong cross-benchmark performance, with 9 of 11 benchmarks showing state-of-the-art results among peers of similar size and a reported 3x speedup on common edge hardware. This work advances edge-ready VLLMs, offering a concrete method to balance accuracy and real-time performance in industrial settings.

Abstract

Recently, due to the advancement of multimodal technology, people are attempting to use visual large language models (VLLMs) in industrial production. Many deep learning models (DLMs) deployed in the production environment are gradually being replaced by VLLMs. Compared with DLMs, VLLMs have some advantages in industrial applications: (1) Their strong generalization ability enables them to perform well across a wide range of tasks. (2) They are flexible and can deal with unfamiliar samples through context learning quickly. However, VLLMs also have obvious drawbacks: (1) VLLMs do not perform as well as custom-developed DLMs in specific domains. (2) The number of parameters in VLLMs is generally quite large, and their deployment requires substantial computational resources. (3) VLLMs generally operate much slower than DLMs, making real-time response challenging to achieve. To better utilize VLLMs in industrial applications, we introduce AuroraEdge-V-2B in this work, a compact, robust, and high-speed VLLM designed for edge deployment. To make the model run faster, we also propose a compression-fusion method to improve inference efficiency. AuroraEdge-V-2B has the following notable features: (1) Easy deployment and faster: It has only 2B parameters and is highly suitable for edge deployment, offering better real-time performance. (2) Fewer visual tokens and cheaper: It significantly reduces the number of visual tokens in the decoding process, thereby reducing the floating-point operations by half during inference and making it cheaper to use. (3) Strong performance: It gets a higher score on 9 benchmarks than models with the same number of parameter (e.g., Qwen2-VL-2B, Qwen2.5-VL-3B, InternVL-2.5-2B).

Figures (9)

• Figure 1: AuroraEdge-V-2B achieves high scores on 11 benchmarks, outperforming other state-of-the-art (SOTA) models with the same parameter scale on 9 of them.
• Figure 2: The LLaVA architecture.
• Figure 3: The overall architecture of AuroraEdge-V-2B. Based on LLaVA, we added a Token Compressor to reduce the number of visual tokens, and incorporated a Fusion Module to compensate for the visual loss caused by compression, thereby enhancing the efficiency of the model.The details of the Token Compressor(see Figure \ref{['fig:compressor']}) and Fusion Module(see Figure \ref{['fig:fusion']}) are introduced in the chapters below.
• Figure 4: The token compressors of AuroraEdge-V-2B. We experimented with three methods—1.Conv2d, 2.MaxPool2d, and 3.MLP—for compressing the visual tokens. Through training experiments on these methods, we observed that the MLP converges the fastest and achieves the best performance for the same number of training steps, leading to its adoption as our final token compressor.
• Figure 5: The fusion modules of AuroraEdge-V-2B. We experimented with three methods for multimodal fusion: 1. Cross (Cross-Attention), 2. Decoder (A single-layer transformer decoder), 3. Combined (Accumulation of the results from Cross and Decoder). Among these approaches, the model performance achieved by method 3 (Combined) was the best, and we adopt it as our final solution.