Shakti-VLMs: Scalable Vision-Language Models for Enterprise AI

TL;DR

Shakti-VLM tackles the data-efficiency challenge in vision-language modeling for enterprise AI by introducing a compact 1B and a larger 4B model that achieve strong multimodal performance with substantially fewer training tokens. The authors propose architectural innovations—QK-Normalization, a hybrid normalization scheme, enhanced 2D Rotary Position Embedding, and dynamic patch sizing—coupled with a three-stage training pipeline that isolates decoder pretraining, aligns vision and language with a frozen decoder, and then fine-tunes end-to-end with RLHF and DPO. The resulting models demonstrate strong OCR, document understanding, chart reasoning, and general multimodal reasoning benchmarks, often matching or surpassing larger models while maintaining efficiency. These findings suggest that careful architectural design and staged training can deliver enterprise-grade multimodal AI with practical data and compute requirements, enabling scalable deployment at the edge and in production pipelines.

Abstract

We introduce Shakti VLM, a family of vision-language models in the capacity of 1B and 4B parameters designed to address data efficiency challenges in multimodal learning. While recent VLMs achieve strong performance through extensive training data, Shakti models leverage architectural innovations to attain competitive results with fewer tokens. Key advancements include QK-Normalization for attention stability, hybrid normalization techniques, and enhanced positional encoding. A three-stage training strategy further optimizes learning efficiency. Evaluations show that Shakti-Shakti-VLM-1B and Shakti-VLM-4B excel in document understanding, Visual Reasoning, OCR extraction, and general multimodal reasoning. Our results highlight that high performance can be achieved through model design and training strategy rather than sheer data volume, making Shakti an efficient solution for enterprise-scale multimodal tasks.

Figures (7)

• Figure 1: Training Loss Curve for Shakti-VLM-1B:The graph shows the loss reduction from around 10 to 1 over 35k steps, with stepwise drops likely due to scheduled learning rate adjustments, reflecting a more complex training trajectory.
• Figure 2: Training Loss Curve for Shakti-VLM-4B – The graph illustrates the steady decline in training loss from approximately 2.8 to 1.8 over 20k steps, indicating stable convergence and effective optimization.
• Figure 3: Comparision of Shakti-1B and Qwen2VL-2B Results on different prompts.
• Figure 4: Comparision of Shakti-4B and Qwen2.5VL-7B Results on different prompts.
• Figure :