Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference

Miao Zhang, Ruixiao Zhang, Jianxin Shi, Hengzhi Wang, Hao Fang, Jiangchuan Liu

TL;DR

This work proposes QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation, and introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration.

Abstract

Video-language models (VLMs) are reshaping video querying services, bringing unified solutions to complex perception and reasoning tasks. However, deploying large VLMs in real-world systems remains challenging due to their high resource demands, and remote-based deployment often results in unacceptable response delays. Although small, locally deployable VLMs offer faster responses, they unavoidably fall short in accuracy. To reconcile this trade-off, we propose QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation. Built upon the highly modular architecture of VLMs, QuickGrasp shares the vision representation across model variants to avoid redundant computation. To maximize system-wide efficiency, QuickGrasp introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration. We implement a prototype of QuickGrasp and evaluate it across multiple video understanding benchmarks. The results show that QuickGrasp matches the accuracy of large VLMs while achieving up to a 12.8x reduction in response delay. QuickGrasp represents a key advancement toward building responsive video querying services for open-world understanding that fully leverage the capabilities of VLMs.

QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference

TL;DR

This work proposes QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation, and introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration.

Abstract

Video-language models (VLMs) are reshaping video querying services, bringing unified solutions to complex perception and reasoning tasks. However, deploying large VLMs in real-world systems remains challenging due to their high resource demands, and remote-based deployment often results in unacceptable response delays. Although small, locally deployable VLMs offer faster responses, they unavoidably fall short in accuracy. To reconcile this trade-off, we propose QuickGrasp, a responsive, quality of service (QoS)-aware system that bridges this gap through a local-first architecture with on-demand edge augmentation. Built upon the highly modular architecture of VLMs, QuickGrasp shares the vision representation across model variants to avoid redundant computation. To maximize system-wide efficiency, QuickGrasp introduces three key designs: accelerated video tokenization, query-adaptive edge augmentation, and delay-aware, accuracy-preserving vision token density configuration. We implement a prototype of QuickGrasp and evaluate it across multiple video understanding benchmarks. The results show that QuickGrasp matches the accuracy of large VLMs while achieving up to a 12.8x reduction in response delay. QuickGrasp represents a key advancement toward building responsive video querying services for open-world understanding that fully leverage the capabilities of VLMs.
Paper Structure (17 sections, 16 equations, 13 figures, 1 algorithm)

This paper contains 17 sections, 16 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Preliminary: Video-Language Models
  4. Identifying the Delay Bottleneck in VLM Inference
  5. Analyzing the Accuracy Gap Across VLM Variants
  6. QuickGrasp: System Design
  7. Accelerated Video Tokenization
  8. Query-Adaptive Collaborative Inference
  9. Delay-Aware and Accuracy-Preserving Configuration
  10. Service Prototype Implementation
  11. Evaluation
  12. Evaluation Setup
  13. Overall Performance Improvements
  14. In-depth Analysis
  15. Ablation Study and System Overhead
  16. ...and 2 more sections

Figures (13)

  • Figure 1: Architecture of the VideoChat-Flash models. The LM can be replaced by smaller or larger ones.
  • Figure 2: Inference delay breakdown for the 2B and 7B VLM variants.
  • Figure 3: Accuracy comparison on different lengths of videos.
  • Figure 4: System Overview.
  • Figure 5: Example demonstrating how pipelined processing mitigates inference delay bottlenecks in VLMs.
  • ...and 8 more figures