NGENT: Next-Generation AI Agents Must Integrate Multi-Domain Abilities to Achieve Artificial General Intelligence

TL;DR

The paper argues that achieving AGI requires next-generation AI agents (NGENT) that integrate cross-domain abilities across text, vision, robotics, and more. It outlines a vision of a 1.5-generation agent balancing IQ tasks with engaging persona as a stepping stone to a full 2.0 NGENT, supported by a training pipeline including Instruction Pre-Training, Iterative Supervised Fine-Tuning with a Persona Style Rewriter, and Direct Preference Optimization. The authors justify this approach through evidence of technological convergence (transformer-based architectures, multi-task and tool learning) and user demand for versatile agents, and present preliminary results showing improved IQ/EQ balance over baselines. They also discuss alternative viewpoints (specialization, safety, incremental evolution, collaboration) and emphasize a safety-conscious, modular path toward general-purpose cross-domain agents that could transform how AI assists across contexts.

Abstract

This paper argues that the next generation of AI agent (NGENT) should integrate across-domain abilities to advance toward Artificial General Intelligence (AGI). Although current AI agents are effective in specialized tasks such as robotics, role-playing, and tool-using, they remain confined to narrow domains. We propose that future AI agents should synthesize the strengths of these specialized systems into a unified framework capable of operating across text, vision, robotics, reinforcement learning, emotional intelligence, and beyond. This integration is not only feasible but also essential for achieving the versatility and adaptability that characterize human intelligence. The convergence of technologies across AI domains, coupled with increasing user demand for cross-domain capabilities, suggests that such integration is within reach. Ultimately, the development of these versatile agents is a critical step toward realizing AGI. This paper explores the rationale for this shift, potential pathways for achieving it.

Figures (2)

• Figure 1: The position of this paper. Similar to how GPT-3 and GPT-4 revolutionized natural language processing by excelling in virtually all NLP tasks, we propose that the next generation of AI agents should demonstrate comparable versatility across a wide range of domains.
• Figure 2: The landscape of foundation model-based AI agents. Current research primarily concentrates on specific domains, such as intelligent assistants (e.g., ChatGPT), role-playing (e.g., C.AI), coding (e.g., CodeLlama roziere2023code and Deepseek-Coder guo2024deepseek), tool usage (e.g., TPTU ruan2023tptu and TPTU-2 kong2024tptu), OS operations (e.g., OS-copilot wu2024copilot), or robotic control (e.g., RT-1 brohan2022rt and RT-2 brohan2023rt). However, we believe that the next generation agents must go beyond individual specializations and instead embody general capabilities aimed at achieving AGI, seamlessly integrating diverse skills to operate across a wide range of tasks and contexts. In this paper, we propose a step towards this goal by introducing the concept of a "1.5-generation agent", which focuses on the critical challenge of integrating intelligence with personification, ultimately paving the way for the development of the next-generation AI agent (NGENT).