A Survey on Collaborative Mechanisms Between Large and Small Language Models

TL;DR

This survey addresses the challenge of deploying powerful LLMs on resource-constrained devices by examining collaboration with smaller LMs. It presents a taxonomy of interaction modes—pipeline, routing, auxiliary enhancement, distillation, and fusion—and maps key enabling technologies such as intelligent routing, inter-model communication, fusion, and context management to on-device needs. The work catalogues diverse applications spanning real-time inference, privacy-preserving local data processing, personalization, offline operation, and energy efficiency, and discusses core challenges including overhead, consistency, evaluation, and security. By outlining future trends toward adaptive, deeply fused, and multimodal/embodied collaboration, the paper positions LLM-SLM collaboration as a foundational paradigm for practical, efficient, and ubiquitous AI.

Abstract

Large Language Models (LLMs) deliver powerful AI capabilities but face deployment challenges due to high resource costs and latency, whereas Small Language Models (SLMs) offer efficiency and deployability at the cost of reduced performance. Collaboration between LLMs and SLMs emerges as a crucial paradigm to synergistically balance these trade-offs, enabling advanced AI applications, especially on resource-constrained edge devices. This survey provides a comprehensive overview of LLM-SLM collaboration, detailing various interaction mechanisms (pipeline, routing, auxiliary, distillation, fusion), key enabling technologies, and diverse application scenarios driven by on-device needs like low latency, privacy, personalization, and offline operation. While highlighting the significant potential for creating more efficient, adaptable, and accessible AI, we also discuss persistent challenges including system overhead, inter-model consistency, robust task allocation, evaluation complexity, and security/privacy concerns. Future directions point towards more intelligent adaptive frameworks, deeper model fusion, and expansion into multimodal and embodied AI, positioning LLM-SLM collaboration as a key driver for the next generation of practical and ubiquitous artificial intelligence.

Figures (7)

• Figure 1: CoGenesis framework structure diagram. 1. Context-aware instructional examples. 2. Context-aware Language Models (LLMs) excel in context awareness but pose privacy risks. 3. On-device specialized Small Language Models (SLMs) prioritize privacy but have lower performance. 4. Collaborative LLMs and SLMs enhance privacy protection and improve performance.
• Figure 2: CITER framework structure diagram. Utilizes a router for collaborative inference between SLM and LLM. The router is trained using routing preferences collected through three scenarios. Scenario 1: SLM generates the correct token, routing preference assigned to SLM. Scenario 2: SLM generates an incorrect token, while LLM generates the correct token, routing preference assigned to LLM. Scenario 3: Neither SLM nor LLM generates the correct token; collaborative reasoning is performed to obtain the complete response for assigning routing preference.
• Figure 3: Iterative training framework of Collab-RAG. The SLM updates its parameters based on the generation quality of the LLM reader. This process iterates multiple times, progressively improving the SLM's decomposition capability.
• Figure 4: Overall architecture of the Hymba model
• Figure 5: Edge-cloud collaborative LLM-SLM architecture