Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

Manoj Vishwanathan, Suvinay Subramanian, Anand Raghunathan

TL;DR

This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms, and identifies a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase.

Abstract

Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.

Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

TL;DR

This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms, and identifies a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase.

Abstract

Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.
Paper Structure (9 sections, 3 figures, 1 table)

This paper contains 9 sections, 3 figures, 1 table.

Table of Contents

  1. Problem Statement
  2. Background and Related Work: Vision-Language-Action (VLA) Models
  3. Methodology
  4. Real Hardware Characterization
  5. Performance Projection and Simulation
  6. Evaluations and Analysis
  7. Real Hardware Characterization
  8. Scaled Models on Future Architectures
  9. Conclusion

Figures (3)

  • Figure 1: VLA System Architecture. The workload consists of a Vision Encoder, a Generation Engine (autoregressive decoding), and an Action Transformer.
  • Figure 2: Performance on current edge platforms. Latency of MolmoAct-7B on Jetson Orin and Jetson Thor.
  • Figure 3: Control Frequency for various edge system configurations Higher memory bandwidth and PIM increase control frequency, but achieving the 10 Hz target for long horizon action generation at larger model sizes requires new innovations and algorithm-system co-design.