When One LLM Drools, Multi-LLM Collaboration Rules

TL;DR

This paper argues that a single general-purpose LLM cannot reliably represent the diverse data, skills, and people encountered in real-world use. It introduces a taxonomy of multi-LLM collaboration across API-, text-, logit-, and weight-level interactions and across pretraining, post-training, and inference stages, outlining concrete advantages such as improved factuality, alignment, efficiency, and privacy. The authors acknowledge limitations of existing methods and propose a roadmap including theory-informed protocols, encapsulation, compatible adoption paths, interpretability, and democratized contribution. By framing multi-LLM collaboration as a modular, compositional approach, the work positions it as a practical route toward compositional intelligence and more inclusive AI development.

Abstract

This position paper argues that in many realistic (i.e., complex, contextualized, subjective) scenarios, one LLM is not enough to produce a reliable output. We challenge the status quo of relying solely on a single general-purpose LLM and argue for multi-LLM collaboration to better represent the extensive diversity of data, skills, and people. We first posit that a single LLM underrepresents real-world data distributions, heterogeneous skills, and pluralistic populations, and that such representation gaps cannot be trivially patched by further training a single LLM. We then organize existing multi-LLM collaboration methods into a hierarchy, based on the level of access and information exchange, ranging from API-level, text-level, logit-level, to weight-level collaboration. Based on these methods, we highlight how multi-LLM collaboration addresses challenges that a single LLM struggles with, such as reliability, democratization, and pluralism. Finally, we identify the limitations of existing multi-LLM methods and motivate future work. We envision multi-LLM collaboration as an essential path toward compositional intelligence and collaborative AI development.

Figures (2)

• Figure 1: Despite the quest for general-purpose models, a single LLM suffers from underrepresentation of data (language varieties, domains, styles), skills (reasoning abilities, linguistic and communication skills, creative capacities, and technical competencies), and people (opinions, values, cultural norms).
• Figure 2: We propose a typology of multi-LLM collaboration approaches, focusing on different levels of access to LLMs, and survey existing methods that fall into each type.