Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Input-Adaptive Visual Preprocessing for Efficient Fast Vision-Language Model Inference

Putu Indah Githa Cahyani, Komang David Dananjaya Suartana, Novanto Yudistira

TL;DR

The paper tackles the deployment bottleneck of vision-language models caused by high-resolution visual inputs. It introduces input-adaptive preprocessing consisting of content-aware analysis, adaptive resolution, and content-aware cropping, implemented as a front-end to FastVLM without retraining. On a DocVQA subset, it achieves over 50% reduction in per-image inference time, 0.1–0.2s faster mean full-generation time, and 55–58% fewer visual tokens, while maintaining high visual quality and stable reasoning. This work demonstrates that lightweight preprocessing can substantially improve deployment-oriented efficiency and serves as a reproducible, complementary approach to architectural optimizations.

Abstract

Vision-Language Models (VLMs) have demonstrated strong performance on multimodal reasoning tasks, but their deployment remains challenging due to high inference latency and computational cost, particularly when processing high-resolution visual inputs. While recent architectures such as FastVLM improve efficiency through optimized vision encoders, existing pipelines still rely on static visual preprocessing, leading to redundant computation for visually simple inputs. In this work, we propose an adaptive visual preprocessing method that dynamically adjusts input resolution and spatial coverage based on image content characteristics. The proposed approach combines content-aware image analysis, adaptive resolution selection, and content-aware cropping to reduce visual redundancy prior to vision encoding. Importantly, the method is integrated with FastVLM without modifying its architecture or requiring retraining. We evaluate the proposed method on a subset of the DocVQA dataset in an inference-only setting, focusing on efficiency-oriented metrics. Experimental results show that adaptive preprocessing reduces per-image inference time by over 50\%, lowers mean full generation time, and achieves a consistent reduction of more than 55\% in visual token count compared to the baseline pipeline. These findings demonstrate that input-aware preprocessing is an effective and lightweight strategy for improving deployment-oriented efficiency of vision-language models. To facilitate reproducibility, our implementation is provided as a fork of the FastVLM repository, incorporating the files for the proposed method, and is available at https://github.com/kmdavidds/mlfastlm.

Input-Adaptive Visual Preprocessing for Efficient Fast Vision-Language Model Inference

TL;DR

The paper tackles the deployment bottleneck of vision-language models caused by high-resolution visual inputs. It introduces input-adaptive preprocessing consisting of content-aware analysis, adaptive resolution, and content-aware cropping, implemented as a front-end to FastVLM without retraining. On a DocVQA subset, it achieves over 50% reduction in per-image inference time, 0.1–0.2s faster mean full-generation time, and 55–58% fewer visual tokens, while maintaining high visual quality and stable reasoning. This work demonstrates that lightweight preprocessing can substantially improve deployment-oriented efficiency and serves as a reproducible, complementary approach to architectural optimizations.

Abstract

Vision-Language Models (VLMs) have demonstrated strong performance on multimodal reasoning tasks, but their deployment remains challenging due to high inference latency and computational cost, particularly when processing high-resolution visual inputs. While recent architectures such as FastVLM improve efficiency through optimized vision encoders, existing pipelines still rely on static visual preprocessing, leading to redundant computation for visually simple inputs. In this work, we propose an adaptive visual preprocessing method that dynamically adjusts input resolution and spatial coverage based on image content characteristics. The proposed approach combines content-aware image analysis, adaptive resolution selection, and content-aware cropping to reduce visual redundancy prior to vision encoding. Importantly, the method is integrated with FastVLM without modifying its architecture or requiring retraining. We evaluate the proposed method on a subset of the DocVQA dataset in an inference-only setting, focusing on efficiency-oriented metrics. Experimental results show that adaptive preprocessing reduces per-image inference time by over 50\%, lowers mean full generation time, and achieves a consistent reduction of more than 55\% in visual token count compared to the baseline pipeline. These findings demonstrate that input-aware preprocessing is an effective and lightweight strategy for improving deployment-oriented efficiency of vision-language models. To facilitate reproducibility, our implementation is provided as a fork of the FastVLM repository, incorporating the files for the proposed method, and is available at https://github.com/kmdavidds/mlfastlm.
Paper Structure (7 sections, 4 figures)

This paper contains 7 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. Experimental Setup
  5. Results
  6. Discussion
  7. Conclusion

Figures (4)

  • Figure 1: Performance comparison of FastVLM and representative efficient vision-language models across varying numbers of visual tokens.
  • Figure 2: High-level overview of the proposed adaptive preprocessing pipeline integrated with FastVLM.
  • Figure 3: Adaptive resolution selection based on image complexity, where edge density or entropy is used to categorize inputs into low, medium, and high complexity levels, followed by dynamic resolution assignment.
  • Figure 4: Content-aware cropping pipeline that identifies informative regions by detecting content pixels, computes the bounding box of relevant areas, and crops the image before forwarding it to the FastVLM backbone.