LLM Applications: Current Paradigms and the Next Frontier

TL;DR

The paper addresses fragmentation and limited interoperability in the burgeoning field of LLM applications by proposing a unified three-layer framework: infrastructure, protocol, and application. It surveys four dominant paradigms—LLM app stores, LLM agents, self-hosted LLM services, and LLM-powered devices—analyzing their architectures, ecosystems, and research trends. Key contributions include a comparative landscape, a next-frontier architecture to mitigate fragmentation, and a forward-looking roadmap emphasizing protocol-driven interoperability, secure-by-design practices, human-centered monitoring, device ubiquity, and composable applications. The work offers practical implications for building open, secure, and scalable LLM ecosystems that span cloud, edge, and device contexts, enabling better cross-platform collaboration and trustworthy intelligent systems.

Abstract

The development of large language models (LLMs) has given rise to four major application paradigms: LLM app stores, LLM agents, self-hosted LLM services, and LLM-powered devices. Each has its advantages but also shares common challenges. LLM app stores lower the barrier to development but lead to platform lock-in; LLM agents provide autonomy but lack a unified communication mechanism; self-hosted LLM services enhance control but increase deployment complexity; and LLM-powered devices improve privacy and real-time performance but are limited by hardware. This paper reviews and analyzes these paradigms, covering architecture design, application ecosystem, research progress, as well as the challenges and open problems they face. Based on this, we outline the next frontier of LLM applications, characterizing them through three interconnected layers: infrastructure, protocol, and application. We describe their responsibilities and roles of each layer and demonstrate how to mitigate existing fragmentation limitations and improve security and scalability. Finally, we discuss key future challenges, identify opportunities such as protocol-driven cross-platform collaboration and device integration, and propose a research roadmap for openness, security, and sustainability.

Figures (5)

• Figure 1: Configuration components of LLM apps. The center section contains the core configuration, including basic information, character settings, scenarios, and example dialogues. Five advanced functions (basic settings, memory, knowledge base, skills, and workflow) are distributed on both sides of the configuration area.
• Figure 2: Architecture of an LLM agent. The agent interacts with its environment by receiving user input and contextual signals. Perception handles multimodal inputs, which are processed in the brain for memory, reasoning, and knowledge retrieval. Actions are then produced as text, external tool usage, or embodied operations, completing the perception–cognition–action loop.
• Figure 3: Architecture of self-hosted LLM service. The process starts from the user interface, where input queries are sent to the model serving framework. Queries may be embedded through an embedding model and stored in a vector database, which supports knowledge retrieval. The inference engine forwards requests to LLMs, runs inference, and generates outputs. Finally, results are returned to the user, completing the service pipeline.
• Figure 4: Deployment modes of LLM-powered devices. Computation can occur in the cloud, where centralized servers process inputs and deliver results to end devices, or at the edge, where local edge servers handle computation closer to the devices.
• Figure 5: Three-layer architecture for the next frontier of LLM applications. The infrastructure layer provides foundational models, computing resources, communication networks, and data sources. The protocol layer coordinates interactions among agents, resources, and tools through standardized frameworks and protocols. The application layer delivers diverse application forms, where users interact with LLM-based systems for perception, planning, memory, and action.