Towards Adaptive Mechanism Activation in Language Agent

TL;DR

ALAMA tackles rigidity in Language Agents by enabling adaptive activation of multiple mechanisms through UniAct, which unifies workflows by explicit Actions. It introduces self-exploration for data-efficient training and two optimization modules, Implicit Mechanism Activation Optimization ($L_{IMAO}$) and Mechanism Activation Adaptability Optimization ($L_{MAAO}$), based on a $KTO$-based objective. Empirical results on mathematical and knowledge-intensive reasoning show ALAMA surpassing fixed-mechanism baselines and strong fine-tuning approaches, with solid generalization to held-out tasks and robustness over majority voting. The work demonstrates a practical, data-efficient path toward more dynamic, context-sensitive Language Agents that can select appropriate solution strategies across varied tasks.

Abstract

Language Agent could be endowed with different mechanisms for autonomous task accomplishment. Current agents typically rely on fixed mechanisms or a set of mechanisms activated in a predefined order, limiting their adaptation to varied potential task solution structures. To this end, this paper proposes \textbf{A}daptive \textbf{L}anguage \textbf{A}gent \textbf{M}echanism \textbf{A}ctivation Learning with Self-Exploration (\textbf{ALAMA}), which focuses on optimizing mechanism activation adaptability without reliance on expert models. Initially, it builds a harmonized agent framework (\textbf{UniAct}) to \textbf{Uni}fy different mechanisms via \textbf{Act}ions. Then it leverages a training-efficient optimization method based on self-exploration to enable the UniAct to adaptively activate the appropriate mechanisms according to the potential characteristics of the task. Experimental results demonstrate significant improvements in downstream agent tasks, affirming the effectiveness of our approach in facilitating more dynamic and context-sensitive mechanism activation.

Figures (4)

• Figure 1: Illustration of Language Agent with different mechanisms. (a). Vanilla agent with fixed mechanisms by In-Context learning. (b). ALAMA with different mechanisms learn to fit into different environments by Self-Exploration.
• Figure 2: The UniAct trajectory examples for five mechanisms. The underlined contents are generated by the vanilla agent or from external feedback.
• Figure 3: The illustration of ALAMA process. The UniAct trajectories are collected by Self-Exploration with manual mechanism activation. For tasks with mechanism sensitivity, we use the corresponding positive trajectories for Implicit Mechanism Activation Optimization, and utilize both positive and negative ones for Mechanism Activation Adaptability Optimization.
• Figure 4: Mechanism specificity analysis results on GSM8K. OLAMA represents oracle mechanism activation, which selects the most appropriate mechanism for each task. Solved-by-All represents that corresponding tasks could be solved by all mechanisms respectively. And Residual represents the performance gap (yellow part) between different mechanisms and Solved-by-All, which shows the specificity.