Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Atom: Efficient On-Device Video-Language Pipelines Through Modular Reuse

Kunjal Panchal, Saayan Mitra, Somdeb Sarkhel, Haoliang Wang, Ishita Dasgupta, Gang Wu, Hui Guan

TL;DR

Atom tackles the challenge of running multi-stage video-language pipelines on mobile devices by introducing a reuse-centric architecture that modularizes a VLM into persistent, shareable components (video encoder and language decoder) and enables parallel execution. By keeping modules loaded and reusing a single strong decoder across subtasks, Atom eliminates repeated model loading and data movement, significantly reducing end-to-end latency while maintaining performance. Quantization with TorchAO and careful architectural choices keep memory usage within typical smartphone RAM (6 GB) and preserve on-device privacy by avoiding cloud inference. The approach generalizes beyond captioning and retrieval to broader video-language tasks, offering a practical, scalable path for edge AI applications.

Abstract

Recent advances in video-language models have enabled powerful applications like video retrieval, captioning, and assembly. However, executing such multi-stage pipelines efficiently on mobile devices remains challenging due to redundant model loads and fragmented execution. We introduce Atom, an on-device system that restructures video-language pipelines for fast and efficient execution. Atom decomposes a billion-parameter model into reusable modules, such as the visual encoder and language decoder, and reuses them across subtasks like captioning, reasoning, and indexing. This reuse-centric design eliminates repeated model loading and enables parallel execution, reducing end-to-end latency without sacrificing performance. On commodity smartphones, Atom achieves 27--33% faster execution compared to non-reuse baselines, with only marginal performance drop ($\leq$ 2.3 Recall@1 in retrieval, $\leq$ 1.5 CIDEr in captioning). These results position Atom as a practical, scalable approach for efficient video-language understanding on edge devices.

Atom: Efficient On-Device Video-Language Pipelines Through Modular Reuse

TL;DR

Atom tackles the challenge of running multi-stage video-language pipelines on mobile devices by introducing a reuse-centric architecture that modularizes a VLM into persistent, shareable components (video encoder and language decoder) and enables parallel execution. By keeping modules loaded and reusing a single strong decoder across subtasks, Atom eliminates repeated model loading and data movement, significantly reducing end-to-end latency while maintaining performance. Quantization with TorchAO and careful architectural choices keep memory usage within typical smartphone RAM (6 GB) and preserve on-device privacy by avoiding cloud inference. The approach generalizes beyond captioning and retrieval to broader video-language tasks, offering a practical, scalable path for edge AI applications.

Abstract

Recent advances in video-language models have enabled powerful applications like video retrieval, captioning, and assembly. However, executing such multi-stage pipelines efficiently on mobile devices remains challenging due to redundant model loads and fragmented execution. We introduce Atom, an on-device system that restructures video-language pipelines for fast and efficient execution. Atom decomposes a billion-parameter model into reusable modules, such as the visual encoder and language decoder, and reuses them across subtasks like captioning, reasoning, and indexing. This reuse-centric design eliminates repeated model loading and enables parallel execution, reducing end-to-end latency without sacrificing performance. On commodity smartphones, Atom achieves 27--33% faster execution compared to non-reuse baselines, with only marginal performance drop ( 2.3 Recall@1 in retrieval, 1.5 CIDEr in captioning). These results position Atom as a practical, scalable approach for efficient video-language understanding on edge devices.

Paper Structure

This paper contains 41 sections, 1 equation, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Design of Atom
  4. Modular Execution of VLM Pipelines
  5. Persistent Module Reuse
  6. Parallelism through Modular Design
  7. Applicability to Other Tasks.
  8. Implementation of Atom
  9. Adaptability to Other VLMs.
  10. Quantization.
  11. Results and Discussion
  12. Experimental Setup
  13. Tasks and Models.
  14. Datasets.
  15. Counterparts for Comparison.
  16. ...and 26 more sections

Figures (3)

  • Figure 1: An end-to-end on-device sequential pipeline for video assembly. Given (i) user videos and (ii) a user prompt, the pipeline converts all the video clips into their text representations, generates a script and assembles selected video clips accordingly.
  • Figure 2: Sequential (left) vs. Atom's modular-parallel (right) pipeline execution.
  • Figure 3: Qualitative analysis of the three tasks: (a) video captioning, (b) video retrieval, (c) video assembly. The results are based on 10 participants. The scale of different metric scores is [0, 10].