Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference

TL;DR

Talaria presents an interactive, hardware-aware visualization system that enables on-device inference optimization through model compilation, dual views of statistics and architecture, and real-time simulation of compression techniques. Grounded in formative research with Apple practitioners, it unifies analytic and geometric perspectives to identify bottlenecks, compare optimizations, and support collaborative workflows. Over 2 years of deployment, Talaria demonstrates broad adoption (800+ users, 3{,}600+ submissions) and strong usability signals for its core views and optimization capabilities, while also outlining future work in model comparison, behavioral metrics integration, and automated code-editing playgrounds. The work highlights the practical impact of integrated visualization and optimization tooling for efficiently deploying neural networks on resource-constrained devices, with potential to streamline iterative, cross-disciplinary collaboration across product teams.

Abstract

On-device machine learning (ML) moves computation from the cloud to personal devices, protecting user privacy and enabling intelligent user experiences. However, fitting models on devices with limited resources presents a major technical challenge: practitioners need to optimize models and balance hardware metrics such as model size, latency, and power. To help practitioners create efficient ML models, we designed and developed Talaria: a model visualization and optimization system. Talaria enables practitioners to compile models to hardware, interactively visualize model statistics, and simulate optimizations to test the impact on inference metrics. Since its internal deployment two years ago, we have evaluated Talaria using three methodologies: (1) a log analysis highlighting its growth of 800+ practitioners submitting 3,600+ models; (2) a usability survey with 26 users assessing the utility of 20 Talaria features; and (3) a qualitative interview with the 7 most active users about their experience using Talaria.

Figures (12)

• Figure 1: An illustration of three common model compression techniques built into Talaria. (A) Quantization converts data types from high-precision formats (e.g., fp32) to low-precision formats (e.g., int8). (B) Pruning/Sparsification removes unnecessary weights from neural networks. (C) Palettization maps model weights to a discrete set of precomputed (or learned) values.
• Figure 2: Five different models visualized in Talaria with increasingly complex architectures.
• Figure 3: Three examples of the Graph View encoding different hardware metrics on the same model to quickly identify potential model bottlenecks. Dark blue nodes indicate higher values for a metric, e.g., latency, memory, or power usage.
• Figure 4: An illustration of two types of model optimization. (A) Model-wide optimization applies a compression technique to the entire model, regardless of outcome. (B) Targeted optimization only compresses certain model operations. Talaria supports both, and allows practitioners to interactively optimize individual model operations.
• Figure 5: Talaria's (A) model-wide optimization for quick experimentation and (B) targeted optimization for compressing a single hardware operation. Targeted optimization displays a table where rows are different compression techniques, with metric changes colored green or red. In this example, a user has filtered the table to only consider optimizations where the input and output formats are quantized to int8.