Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions

TL;DR

Collaborative learning between on device small models and cloud large models addresses latency cost and privacy bottlenecks of cloud centric systems. The paper surveys a layered framework from hardware to application, classifies collaboration into data based feature based and parameter based approaches, and reviews advances in learning engines task splitting distillation pruning quantization and federated learning. It also covers datasets benchmarks and real world deployments in CV NLP recommender livestreaming and personal assistants, highlighting practical deployment considerations and open research directions. The work underscores the need for theory grounded analyses universal yet specialized system designs and scalable real device platforms to realize efficient personalized intelligent services.

Abstract

The conventional cloud-based large model learning framework is increasingly constrained by latency, cost, personalization, and privacy concerns. In this survey, we explore an emerging paradigm: collaborative learning between on-device small model and cloud-based large model, which promises low-latency, cost-efficient, and personalized intelligent services while preserving user privacy. We provide a comprehensive review across hardware, system, algorithm, and application layers. At each layer, we summarize key problems and recent advances from both academia and industry. In particular, we categorize collaboration algorithms into data-based, feature-based, and parameter-based frameworks. We also review publicly available datasets and evaluation metrics with user-level or device-level consideration tailored to collaborative learning settings. We further highlight real-world deployments, ranging from recommender systems and mobile livestreaming to personal intelligent assistants. We finally point out open research directions to guide future development in this rapidly evolving field.

Figures (3)

• Figure 1: Learning paradigms comparison.
• Figure 2: Overall framework.
• Figure 3: Key problems at different layers.