Practical Considerations for Agentic LLM Systems
Chris Sypherd, Vaishak Belle
TL;DR
This paper surveys practical considerations for deploying agentic LLM systems, organizing guidance around four pillars—Planning, Memory, Tools, and Control Flow—to bridge academia and industry. It argues that LLM-based planning is often weak and should be complemented with task decomposition and external planning tools, while grounding and memory (RAG and long-term memory) are essential for robust context management. The work also covers tool design and management, structuring control flow with reliable error handling and stopping mechanisms, and integrating with traditional engineering practices, including evaluation frameworks and short-circuiting. By combining these elements, the paper provides actionable recommendations for building robust, real-world LLM agents and highlights key evaluation and deployment challenges as avenues for future work.
Abstract
As the strength of Large Language Models (LLMs) has grown over recent years, so too has interest in their use as the underlying models for autonomous agents. Although LLMs demonstrate emergent abilities and broad expertise across natural language domains, their inherent unpredictability makes the implementation of LLM agents challenging, resulting in a gap between related research and the real-world implementation of such systems. To bridge this gap, this paper frames actionable insights and considerations from the research community in the context of established application paradigms to enable the construction and facilitate the informed deployment of robust LLM agents. Namely, we position relevant research findings into four broad categories--Planning, Memory, Tools, and Control Flow--based on common practices in application-focused literature and highlight practical considerations to make when designing agentic LLMs for real-world applications, such as handling stochasticity and managing resources efficiently. While we do not conduct empirical evaluations, we do provide the necessary background for discussing critical aspects of agentic LLM designs, both in academia and industry.