A Functional Software Reference Architecture for LLM-Integrated Systems
Alessio Bucaioni, Martin Weyssow, Junda He, Yunbo Lyu, David Lo
TL;DR
This paper addresses the lack of a standard software reference architecture for LLM-integrated systems and proposes a preliminary functional RA with four layered architecture (Presentation, Application logic, LLM integration, Data management) plus Monitoring and Guardrail sidecars to manage controls and policy enforcement. It identifies key architectural concerns from literature and practice, and validates the RA by mapping these concerns and testing against three open-source systems (MaxKB, Continue, InternVL) across computer vision, text processing, and coding. The contributions include a structured RA design, cross-layer monitoring and governance concepts, and a practical validation pathway, aiming to guide design, evaluation, and evolution of LLM-enabled software. This work aims to facilitate modularity, interoperability, privacy, and safety in increasingly complex LLM-driven systems, with a roadmap for broader empirical validation and architectural refinement.
Abstract
The integration of large language models into software systems is transforming capabilities such as natural language understanding, decision-making, and autonomous task execution. However, the absence of a commonly accepted software reference architecture hinders systematic reasoning about their design and quality attributes. This gap makes it challenging to address critical concerns like privacy, security, modularity, and interoperability, which are increasingly important as these systems grow in complexity and societal impact. In this paper, we describe our \textit{emerging} results for a preliminary functional reference architecture as a conceptual framework to address these challenges and guide the design, evaluation, and evolution of large language model-integrated systems. We identify key architectural concerns for these systems, informed by current research and practice. We then evaluate how the architecture addresses these concerns and validate its applicability using three open-source large language model-integrated systems in computer vision, text processing, and coding.