Optimizing Storage Overhead of User Behavior Log for ML-embedded Mobile Apps

TL;DR

This paper tackles the storage bottleneck of on-device behavior logs used to power ML in mobile apps. It introduces AdaLog, a lightweight shim that combines feature-level data merging via a hierarchical hypergraph-based approach, behavior-level log splitting through virtually hashed attribute naming, and an incremental update mechanism to adapt to evolving user behavior, all while preserving inference accuracy and latency. The merging algorithm runs in $O(|\mathcal{F}|^3)$ and is implemented as on-device computations, enabling substantial reductions in log size (up to 44% on average) with modest resource usage (roughly 2 seconds of overhead and under 20 MB memory). Empirical evaluation across multiple app domains shows that AdaLog enables more on-device ML services under the same storage budget, improving scalability and user experience for ML-enabled mobile apps.

Abstract

Machine learning (ML) models are increasingly integrated into modern mobile apps to enable personalized and intelligent services. These models typically rely on rich input features derived from historical user behaviors to capture user intents. However, as ML-driven services become more prevalent, recording necessary user behavior data imposes substantial storage cost on mobile apps, leading to lower system responsiveness and more app uninstalls. To address this storage bottleneck, we present AdaLog, a lightweight and adaptive system designed to improve the storage efficiency of user behavior log in ML-embedded mobile apps, without compromising model inference accuracy or latency. We identify two key inefficiencies in current industrial practices of user behavior log: (i) redundant logging of overlapping behavior data across different features and models, and (ii) sparse storage caused by storing behaviors with heterogeneous attribute descriptions in a single log file. To solve these issues, AdaLog first formulates the elimination of feature-level redundant data as a maximum weighted matching problem in hypergraphs, and proposes a hierarchical algorithm for efficient on-device deployment. Then, AdaLog employs a virtually hashed attribute design to distribute heterogeneous behaviors into a few log files with physically dense storage. Finally, to ensure scalability to dynamic user behavior patterns, AdaLog designs an incremental update mechanism to minimize the I/O operations needed for adapting outdated behavior log. We implement a prototype of AdaLog and deploy it into popular mobile apps in collaboration with our industry partner. Evaluations on real-world user data show that AdaLog reduces behavior log size by 19% to 44% with minimal system overhead (only 2 seconds latency and 15 MB memory usage), providing a more efficient data foundation for broader adoption of on-device ML.

Figures (21)

• Figure 1: In mobile apps like TikTok, behavior log acts as a middle layer between user behaviors and ML-driven services: (i) Each user behavior is captured as an event containing multiple attributes for description; (ii) Different features apply distinct filters to judge event relevance and select necessary attributes to store in behavior log, consisting of event data and index structure; (iii) During real-time feature computation, each feature uses index structure and time window constraints to retrieve relevant rows for accurate computation.
• Figure 2: Statistics of real-world mobile data to illustrate behavior log storage bottleneck: (a) Each single ML model logs numerous behavior types and event rows daily; (b) Deploying more ML models linearly increases the storage cost due to their heterogeneous logging requirements.
• Figure 3: Storage redundancy and sparsity of behavior log for an average user, where x-axis denotes different behavior types and y-axis depicts their redundancy and sparsity.
• Figure 4: AdaLog architecture and workflow within on-device ML pipelines.
• Figure 5: Comparing behavior logs before and after conducting feature-level data merging.