Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thought Communication in Multiagent Collaboration

Yujia Zheng, Zhuokai Zhao, Zijian Li, Yaqi Xie, Mingze Gao, Lizhu Zhang, Kun Zhang

TL;DR

This work addresses the limits of language-based collaboration by proposing latent thought communication in multiagent systems. It models agent states as $H_t = f(Z_t)$ with latent thoughts $Z_t$ drawn from $P_z$, and provides nonparametric identifiability guarantees for recovering both shared and private thoughts and their incidence structure among agents. Building on this theory, it introduces ThoughtComm, a practical framework that (i) encodes agent states into a sparsity-regularized latent space, (ii) routes latent thoughts according to a learned structure, and (iii) injects content via per-agent prefixes to guide next-round generation. Empirically, ThoughtCommOutperforms baselines on synthetic identifiability tests and real-world math-reasoning benchmarks (MATH, GSM8K) across diverse models, while maintaining scalability and modest training overhead via a lightweight autoencoder and adapter, suggesting broad applicability beyond LLMs and across modalities.

Abstract

Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints, most LLM-based multi-agent systems still rely solely on natural language, exchanging tokens or their embeddings. To go beyond language, we introduce a new paradigm, thought communication, which enables agents to interact directly mind-to-mind, akin to telepathy. To uncover these latent thoughts in a principled way, we formalize the process as a general latent variable model, where agent states are generated by an unknown function of underlying thoughts. We prove that, in a nonparametric setting without auxiliary information, both shared and private latent thoughts between any pair of agents can be identified. Moreover, the global structure of thought sharing, including which agents share which thoughts and how these relationships are structured, can also be recovered with theoretical guarantees. Guided by the established theory, we develop a framework that extracts latent thoughts from all agents prior to communication and assigns each agent the relevant thoughts, along with their sharing patterns. This paradigm naturally extends beyond LLMs to all modalities, as most observational data arise from hidden generative processes. Experiments on both synthetic and real-world benchmarks validate the theory and demonstrate the collaborative advantages of thought communication. We hope this work illuminates the potential of leveraging the hidden world, as many challenges remain unsolvable through surface-level observation alone, regardless of compute or data scale.

Thought Communication in Multiagent Collaboration

TL;DR

This work addresses the limits of language-based collaboration by proposing latent thought communication in multiagent systems. It models agent states as with latent thoughts drawn from , and provides nonparametric identifiability guarantees for recovering both shared and private thoughts and their incidence structure among agents. Building on this theory, it introduces ThoughtComm, a practical framework that (i) encodes agent states into a sparsity-regularized latent space, (ii) routes latent thoughts according to a learned structure, and (iii) injects content via per-agent prefixes to guide next-round generation. Empirically, ThoughtCommOutperforms baselines on synthetic identifiability tests and real-world math-reasoning benchmarks (MATH, GSM8K) across diverse models, while maintaining scalability and modest training overhead via a lightweight autoencoder and adapter, suggesting broad applicability beyond LLMs and across modalities.

Abstract

Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints, most LLM-based multi-agent systems still rely solely on natural language, exchanging tokens or their embeddings. To go beyond language, we introduce a new paradigm, thought communication, which enables agents to interact directly mind-to-mind, akin to telepathy. To uncover these latent thoughts in a principled way, we formalize the process as a general latent variable model, where agent states are generated by an unknown function of underlying thoughts. We prove that, in a nonparametric setting without auxiliary information, both shared and private latent thoughts between any pair of agents can be identified. Moreover, the global structure of thought sharing, including which agents share which thoughts and how these relationships are structured, can also be recovered with theoretical guarantees. Guided by the established theory, we develop a framework that extracts latent thoughts from all agents prior to communication and assigns each agent the relevant thoughts, along with their sharing patterns. This paradigm naturally extends beyond LLMs to all modalities, as most observational data arise from hidden generative processes. Experiments on both synthetic and real-world benchmarks validate the theory and demonstrate the collaborative advantages of thought communication. We hope this work illuminates the potential of leveraging the hidden world, as many challenges remain unsolvable through surface-level observation alone, regardless of compute or data scale.
Paper Structure (42 sections, 6 theorems, 85 equations, 10 figures, 1 table)

This paper contains 42 sections, 6 theorems, 85 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Data-generating process.
  4. The structure of thoughts.
  5. Identifiability Theory
  6. Identifiability of Shared Thoughts
  7. Interpretation and discussion.
  8. Assumption.
  9. Identifiability of Private Thoughts
  10. Interpretation and discussion.
  11. The Structure of Thoughts
  12. Interpretation and discussion.
  13. Discussion on Theoretical Contribution
  14. ThoughtComm: Multiagent Communication via Thought
  15. Uncovering the Latent Thoughts
  16. ...and 27 more sections

Key Result

Theorem 1

Suppose that for each $i \in [n_x]$, there exist points where the Jacobians $J_f(Z_t)_{i,\cdot}$ span the support subspace $\mathcal{S}_{{J_f}_{i,\cdot}}$, and that $(J_f(Z_t) \mathcal{M})_{i,\cdot} \in \mathcal{S}_{{J_{\hat{f}}}_{i,\cdot}}$ at those points. If $H_t \overset{d}{=} \hat{f}(\hat{Z}_t)

Figures (10)

  • Figure 1: Each agent answers the same question by selecting a subset of latent thoughts $Z_t$. Agent 1 chooses a car based on carrying luggage , while Agent 2 selects a train for schedule punctuality . Both share the thought of speed .
  • Figure 2: Overview of ThoughtComm. At each communication round $t$, agents encode their model states $H_t^{(i)}$ into a shared latent space via a sparsity-regularized autoencoder, yielding latent thoughts $\hat{Z}_t$. Each dimension $\hat{Z}_{t,j}$ is selectively routed to relevant agents based on the recovered dependency structure, allowing agents to identify both shared and private thoughts for reasoning. The corresponding latent thoughts are then injected into each agent model via prefix adaptation to guide the next response. These updated responses form the input to the next round, enabling multi-agent collaboration beyond purely message exchange.
  • Figure 3: $R^2$ of two models.
  • Figure 4: MCC across setups.
  • Figure 5: Two-agent ThoughtComm with accuracy (solid) and consensus (dashed) performance on MATH hendrycks2021measuring as prefix length increases from 1 to 16.
  • ...and 5 more figures

Theorems & Definitions (11)

  • Example 1
  • Theorem 1: Identifying the shared thoughts
  • Theorem 2: Identifying the private thoughts
  • Theorem 3: Identifying the structure of thoughts
  • Remark 1
  • Theorem 3: Identifying the shared thoughts
  • proof
  • Theorem 3: Identifying the private thoughts
  • proof
  • Theorem 3: Identifying the structure of thoughts
  • ...and 1 more