Architectural Classification of XR Workloads: Cross-Layer Archetypes and Implications

TL;DR

This work addresses the mismatch between diverse XR workloads and CNN-centric accelerators by delivering a cross-layer architectural classification for XR pipelines. It combines a fast, analytical design-space exploration framework with empirical GPU/CPU profiling to identify cross-layer workload archetypes and their sensitivity to on-chip capacity, bandwidth, and phase-based overheads. The study analyzes 12 XR kernels and derives four archetypes (and a phase overlay) with actionable implications, advocating phase-aware scheduling, elastic resource allocation, and explicit staging for irregular workloads. The findings suggest XR performance and energy efficiency hinge on threshold effects and phase dynamics rather than linear gains from generic scaling, informing next-generation XR SoCs that balance capacity, dataflow, and irregularity-aware primitives for real-time edge inference.

Abstract

Edge and mobile platforms for augmented and virtual reality, collectively referred to as extended reality (XR) must deliver deterministic ultra-low-latency performance under stringent power and area constraints. However, the diversity of XR workloads is rapidly increasing, characterized by heterogeneous operator types and complex dataflow structures. This trend poses significant challenges to conventional accelerator architectures centered around convolutional neural networks (CNNs), resulting in diminishing returns for traditional compute-centric optimization strategies. Despite the importance of this problem, a systematic architectural understanding of the full XR pipeline remains lacking. In this paper, we present an architectural classification of XR workloads using a cross-layer methodology that integrates model-based high-level design space exploration (DSE) with empirical profiling on commercial GPU and CPU hardware. By analyzing a representative set of workloads spanning 12 distinct XR kernels, we distill their complex architectural characteristics into a small set of cross-layer workload archetypes (e.g., capacity-limited and overhead-sensitive). Building on these archetypes, we further extract key architectural insights and provide actionable design guidelines for next-generation XR SoCs. Our study highlights that XR architecture design must shift from generic resource scaling toward phase-aware scheduling and elastic resource allocation in order to achieve greater energy efficiency and high performance in future XR systems.

Figures (7)

• Figure 1: Near-memory SIMD Modeling Template Dataflow Overview
• Figure 2: L1-resident tiling and stationary execution flow on the SIMD engine(Conv layer case).
• Figure 3: Automatic L1 tiling for convolution layers. (a) Conv tile variables and the L1 capacity constraint from input/weight/output footprints. (b) Simulated annealing search that explores feasible tiles under the constraint and returns a low-cost tiling. All SA hyperparameters ($T_0$, $T_{\min}$, $\alpha_T$, $L$, and $\delta$) are fixed globally across all workloads.
• Figure 4: Energy vs. on-chip capacity: breakdown and L1–LLC capacity across workloads.
• Figure 5: Pareto fronts of energy vs memory-side latency(Workload from left to right: LoFTR, NeRF, PWC-Net, and Monodepth2).