Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput

Bo Zhang, Shuo Li, Runhe Tian, Yang Yang, Jixin Tang, Jinhao Zhou, Lin Ma

TL;DR

This work tackles the challenge of deploying Vision-Language Models in real-time settings by targeting ultra-low latency and high throughput without sacrificing accuracy. It presents Flash-VL-2B, a ViT-Adapter-LLM architecture that uses a compact SigLIP2 visual encoder, a token-compression adapter with PixelShuffle, and an LLM (Qwen-2.5-1.5B), augmented by Implicit Semantic Stitching (ISS) to maintain semantic continuity across image tiles. A five-stage training pipeline, including Direct Preference Optimization (DPO) with LoRA and diverse data sourcing from InfinityMM, yields strong performance across 11 benchmarks while delivering up to 60.73 tokens/s throughput and sub-0.2 s latency per token, establishing a new Pareto frontier for 2B-size VLMs. The results demonstrate robust gains in multimodal reasoning and OCR-heavy tasks, with ISS and DPO contributing significant improvements, and provide practical guidance for deploying fast, open-source multimodal systems in constrained environments.

Abstract

In this paper, we introduce Flash-VL 2B, a novel approach to optimizing Vision-Language Models (VLMs) for real-time applications, targeting ultra-low latency and high throughput without sacrificing accuracy. Leveraging advanced architectural enhancements and efficient computational strategies, Flash-VL 2B is designed to maximize throughput by reducing processing time while maintaining competitive performance across multiple vision-language benchmarks. Our approach includes tailored architectural choices, token compression mechanisms, data curation, training schemes, and a novel image processing technique called implicit semantic stitching that effectively balances computational load and model performance. Through extensive evaluations on 11 standard VLM benchmarks, we demonstrate that Flash-VL 2B achieves state-of-the-art results in both speed and accuracy, making it a promising solution for deployment in resource-constrained environments and large-scale real-time applications.

Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput

TL;DR

This work tackles the challenge of deploying Vision-Language Models in real-time settings by targeting ultra-low latency and high throughput without sacrificing accuracy. It presents Flash-VL-2B, a ViT-Adapter-LLM architecture that uses a compact SigLIP2 visual encoder, a token-compression adapter with PixelShuffle, and an LLM (Qwen-2.5-1.5B), augmented by Implicit Semantic Stitching (ISS) to maintain semantic continuity across image tiles. A five-stage training pipeline, including Direct Preference Optimization (DPO) with LoRA and diverse data sourcing from InfinityMM, yields strong performance across 11 benchmarks while delivering up to 60.73 tokens/s throughput and sub-0.2 s latency per token, establishing a new Pareto frontier for 2B-size VLMs. The results demonstrate robust gains in multimodal reasoning and OCR-heavy tasks, with ISS and DPO contributing significant improvements, and provide practical guidance for deploying fast, open-source multimodal systems in constrained environments.

Abstract

In this paper, we introduce Flash-VL 2B, a novel approach to optimizing Vision-Language Models (VLMs) for real-time applications, targeting ultra-low latency and high throughput without sacrificing accuracy. Leveraging advanced architectural enhancements and efficient computational strategies, Flash-VL 2B is designed to maximize throughput by reducing processing time while maintaining competitive performance across multiple vision-language benchmarks. Our approach includes tailored architectural choices, token compression mechanisms, data curation, training schemes, and a novel image processing technique called implicit semantic stitching that effectively balances computational load and model performance. Through extensive evaluations on 11 standard VLM benchmarks, we demonstrate that Flash-VL 2B achieves state-of-the-art results in both speed and accuracy, making it a promising solution for deployment in resource-constrained environments and large-scale real-time applications.
Paper Structure (23 sections, 1 equation, 7 figures, 10 tables)

This paper contains 23 sections, 1 equation, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Flash-VL
  4. The VLM Architecture
  5. Data Choices
  6. Training Strategies
  7. Experiments
  8. Setup
  9. Comparison of state-of-the-art lightweight MLLMs
  10. Comparison of ViTs
  11. Ablation Study of DPO
  12. Multi-stage Training
  13. Comparison of Different Image Cropping Strategies
  14. Comparison of Multimodal Reasoning Capacity
  15. Comparison on Latency and Throughput
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Left: Pareto front of VLMs in terms of the average accuracy on 11 standard VLM benchmarks and the TPS (tokens per second). Right: The distribution of the input token numbers processed by the LLM component of each VLM given various image sizes of the MMMU dataset.
  • Figure 2: The Flash-VL architecture adopts a ViT-Adapter-LLM paradigm trained in multi-stages, striking an outstanding trade-off between accuracy and speed.
  • Figure 3: Image processing strategies in VLMs. (a) Static Method (b) Dynamic Cropping Method (c) Dynamic Overlapping Cropping Method (d) The Proposed Implicit Semantic Stitching.
  • Figure 4: Implicit Semantic Stitching. Left: An example of one image tile and its feature map, where the yellow box represents the repetitive areas. The gray area represents the overlapping tokens, and the green ones represent retained tokens. Right: The overlap rate is approximately 12.5%; The number of image tokens of each image tile is 576.
  • Figure 5: Pie chart of the Stage 4-10M data categorized by General, Code, OCR, Math, Chart, Caption, Text, and Special enhancement.
  • ...and 2 more figures