Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication

Weize Chen, Chenfei Yuan, Jiarui Yuan, Yusheng Su, Chen Qian, Cheng Yang, Ruobing Xie, Zhiyuan Liu, Maosong Sun

TL;DR

The paper challenges the default reliance on natural language for LLM reasoning and inter-agent communication by introducing AutoForm, a prompting framework that enables LLMs to autonomously select non-NL formats. Across diverse reasoning and multi-agent tasks, AutoForm yields consistent gains in reasoning efficiency and substantial token reductions in communication, while maintaining or improving effectiveness. The study demonstrates format generalization from task-level cues, transferability across models, and a notable alignment between AutoForm formats and traditional ACLs, offering a path toward more efficient, structured, and scalable LLM collaboration. These findings highlight the potential of non-NL representations to augment LLM capabilities in both single-agent and multi-agent settings, with practical implications for deployment and interoperability.

Abstract

Natural language (NL) has long been the predominant format for human cognition and communication, and by extension, has been similarly pivotal in the development and application of Large Language Models (LLMs). Yet, besides NL, LLMs have seen various non-NL formats during pre-training, such as code and logical expression. NL's status as the optimal format for LLMs, particularly in single-LLM reasoning and multi-agent communication, has not been thoroughly examined. In this work, we challenge the default use of NL by exploring the utility of non-NL formats in these contexts. We show that allowing LLMs to autonomously select the most suitable format before reasoning or communicating leads to a 3.3 to 5.7\% improvement in reasoning efficiency for different LLMs, and up to a 72.7\% reduction in token usage in multi-agent communication, all while maintaining communicative effectiveness. Our comprehensive analysis further reveals that LLMs can devise a format from limited task instructions and that the devised format is effectively transferable across different LLMs. Intriguingly, the structured communication format decided by LLMs exhibits notable parallels with established agent communication languages, suggesting a natural evolution towards efficient, structured communication in agent communication. Our code is released at \url{https://github.com/thunlp/AutoForm}.

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication

TL;DR

The paper challenges the default reliance on natural language for LLM reasoning and inter-agent communication by introducing AutoForm, a prompting framework that enables LLMs to autonomously select non-NL formats. Across diverse reasoning and multi-agent tasks, AutoForm yields consistent gains in reasoning efficiency and substantial token reductions in communication, while maintaining or improving effectiveness. The study demonstrates format generalization from task-level cues, transferability across models, and a notable alignment between AutoForm formats and traditional ACLs, offering a path toward more efficient, structured, and scalable LLM collaboration. These findings highlight the potential of non-NL representations to augment LLM capabilities in both single-agent and multi-agent settings, with practical implications for deployment and interoperability.

Abstract

Natural language (NL) has long been the predominant format for human cognition and communication, and by extension, has been similarly pivotal in the development and application of Large Language Models (LLMs). Yet, besides NL, LLMs have seen various non-NL formats during pre-training, such as code and logical expression. NL's status as the optimal format for LLMs, particularly in single-LLM reasoning and multi-agent communication, has not been thoroughly examined. In this work, we challenge the default use of NL by exploring the utility of non-NL formats in these contexts. We show that allowing LLMs to autonomously select the most suitable format before reasoning or communicating leads to a 3.3 to 5.7\% improvement in reasoning efficiency for different LLMs, and up to a 72.7\% reduction in token usage in multi-agent communication, all while maintaining communicative effectiveness. Our comprehensive analysis further reveals that LLMs can devise a format from limited task instructions and that the devised format is effectively transferable across different LLMs. Intriguingly, the structured communication format decided by LLMs exhibits notable parallels with established agent communication languages, suggesting a natural evolution towards efficient, structured communication in agent communication. Our code is released at \url{https://github.com/thunlp/AutoForm}.
Paper Structure (24 sections, 2 equations, 4 figures, 77 tables)

This paper contains 24 sections, 2 equations, 4 figures, 77 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Agent Problem Solving.
  4. Method
  5. Problem Formulation
  6. Format Choosing for LLMs
  7. Experiments
  8. Experimental Settings
  9. RQ1: The Capability of LLMs on Selecting Suitable Format
  10. RQ2: Formats Chosen in Single-LLM Reasoning
  11. RQ3: Generalization of Format Selection Based on Task Inputs
  12. RQ4: Transferability of Format Across Different LLMs
  13. RQ5: Features of Communication Format
  14. RQ6: Alignment with Conventional Agent Communication Languages
  15. Conclusion
  16. ...and 9 more sections

Figures (4)

  • Figure 1: LLMs may leverage non-NL thought format.
  • Figure 2: Overview of single-LLM reasoning and multi-agent communication using plain CoT versus the CoT with AutoForm. The left side depicts the shift from natural language to alternative formats in single-LLM reasoning, while the right side illustrates the enhanced efficiency in multi-agent communication.
  • Figure 3: Format distribution chosen by Gemini Pro (a), GPT-3.5 (b) and GPT-4 (c), and the overall format distribution across tasks from both models (d).
  • Figure 4: Multi-agent communication examples. The top panel illustrates a traditional natural language conversation, and the bottom panel shows a conversation using AutoForm. Necessary information related to the question is marked in green, redundant information is marked in red, and speech-act-related phrases are marked in orange.