Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
Yihong Tang, Kehai Chen, Liang Yue, Jinxin Fan, Caishen Zhou, Xiaoguang Li, Yuyang Zhang, Mingming Zhao, Shixiong Kai, Kaiyang Guo, Xingshan Zeng, Wenjing Cun, Lifeng Shang, Min Zhang
TL;DR
This survey investigates how LLM-based industry agents evolve through a five-level capability maturity framework by examining three core technologies—Memory, Planning, and Tool Use—and their impact on practical applications across domains. It synthesizes technical foundations, real-world deployment patterns, and diverse evaluation benchmarks, highlighting the sim-to-real gap and the central role of simulation environments for trustworthy adoption. Key contributions include a structured maturity framework linking technology evolution to industry practices, a systematic review of domain-specific benchmarks, and a discussion of governance, safety, and organizational challenges that shape the path toward scalable, reliable industry agents. The work provides a roadmap for building next-generation agents that combine domain knowledge with automated decision-making to drive productivity and innovation in industry. Overall, it emphasizes reliability, specialization, and human-agent collaboration as essential for realizing the transformative potential of LLM-driven industry agents.
Abstract
With the rise of large language models (LLMs), LLM agents capable of autonomous reasoning, planning, and executing complex tasks have become a frontier in artificial intelligence. However, how to translate the research on general agents into productivity that drives industry transformations remains a significant challenge. To address this, this paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on LLMs. Using an industry agent capability maturity framework, it outlines the evolution of agents in industry applications, from "process execution systems" to "adaptive social systems." First, we examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use. We discuss how these technologies evolve from supporting simple tasks in their early forms to enabling complex autonomous systems and collective intelligence in more advanced forms. Then, we provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation. Additionally, this paper reviews the evaluation benchmarks and methods for both fundamental and specialized capabilities, identifying the challenges existing evaluation systems face regarding authenticity, safety, and industry specificity. Finally, we focus on the practical challenges faced by industry agents, exploring their capability boundaries, developmental potential, and governance issues in various scenarios, while providing insights into future directions. By combining technological evolution with industry practices, this review aims to clarify the current state and offer a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.