Aligned Vector Quantization for Edge-Cloud Collabrative Vision-Language Models

TL;DR

This paper introduces an edge-cloud collaborative VQA system, called LLaVA-AlignedVQ, which features a novel Aligned Vector Quantization algorithm (AlignedVQ) that efficiently compress intermediate features without compromising accuracy to support partitioned execution.

Abstract

Vision Language Models (VLMs) are central to Visual Question Answering (VQA) systems and are typically deployed in the cloud due to their high computational demands. However, this cloud-only approach underutilizes edge computational resources and requires significant bandwidth for transmitting raw images. In this paper, we introduce an edge-cloud collaborative VQA system, called LLaVA-AlignedVQ, which features a novel Aligned Vector Quantization algorithm (AlignedVQ) that efficiently compress intermediate features without compromising accuracy to support partitioned execution. Our experiments demonstrate that LLaVA-AlignedVQ achieves approximately 1365x compression rate of intermediate features, reducing data transmission overhead by 96.8% compared to transmitting JPEG90-compressed images to the cloud. LLaVA-AlignedVQ achieves an inference speedup of 2-15x while maintaining high accuracy, remaining within -2.23% to +1.6% of the original model's accuracy performance across eight VQA datasets, compared to the cloud-only solution.

Figures (7)

• Figure 1: Accuracy with compressed data size for different variants of LLaVA on the VQA-v2 dataset. Our approach, LLaVA-AlignedVQ (marked by the red star), achieves a high accuracy of 79.98% with a minimal transmission size of 0.85KB, balancing accuracy and compression efficiency.
• Figure 2: Accuracy performance of LLAVA on VQA-v2 datasets with different VQ variants. (a) Residual VQ with different codebooks (b) Grouped VQ with different number of groups, and (c) Vanilla VQ (blue) and the combined VQ with 3 codebooks and 8 groups (green). LLaVA-Ori: the baseline, i.e., the original LLaVA accuracy. VQ_n$p$_g$q$$@$blockX: the accuracy when using VQ with $p$ codebook and $q$ group after the $X$th transformer block. $n\times$ in red: the multiple by which the compressed data size increases as the number of codebook and groups grows.
• Figure 3: Overview of the Vision Language Model (VLM) with the proposed AlignedVQ. The model consists of a Vision Encoder, a proposed AlignedVQ module, a Projector, and a Language Model. During training, continuous features from the Vision Encoder are quantized with AlignedVQ and iteratively updated with back-propagation. When deployment, only quantized indices are transmitted from edge to cloud, minimizing data transfer while preserving high accuracy.
• Figure 4: Histograms of feature magnitudes for different locations within a transformer block. The normalized layers (LN1 and LN2) have lower coefficient of variation (CV) values, indicating tighter distributions around the mean, making quantization more effective. In contrast, intermediate features from ATTN and FFN layers show higher variability.
• Figure 5: The trade-off between VQA task accuracy and compressed data size on the VQA-v2 dataset.