Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reference Architecture for Designing Foundation Model based Systems

Qinghua Lu, Liming Zhu, Xiwei Xu, Zhenchang Xing, Jon Whittle

TL;DR

This paper tackles the lack of systematic architectural guidance for foundation-model (FM) based systems in the context of responsible AI and evolving interface boundaries. It proposes a pattern-oriented reference architecture with three layers (system, operation, supply chain) and seven key design decisions to balance FM capabilities with governance, safety, and maintainability. The architecture is empirically evaluated by mapping to a responsible AI chatbot, incorporating retrieval augmented generation, verifiers, think-aloud explanations, multi-agent orchestration, and an AIBOM registry. The work offers concrete, actionable guidance for building adaptable, responsible FM-based systems and points toward a pattern catalogue to expand design coverage over time.

Abstract

The release of ChatGPT, Gemini, and other large language model has drawn huge interests on foundations models. There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems. However, there is a lack of systematic guidance on the architecture design. Particularly, the the rapidly growing capabilities of foundations models can eventually absorb other components of AI systems, posing challenges of moving boundary and interface evolution in architecture design. Furthermore, incorporating foundations models into AI systems raises significant concerns about responsible and safe AI due to their opaque nature and rapidly advancing intelligence. To address these challenges, the paper first presents an architecture evolution of AI systems in the era of foundation models, transitioning from "foundation-model-as-a-connector" to "foundation-model-as-a-monolithic architecture". The paper then identifies key design decisions and proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems. The patterns can enable the potential of foundation models while ensuring associated risks.

A Reference Architecture for Designing Foundation Model based Systems

TL;DR

This paper tackles the lack of systematic architectural guidance for foundation-model (FM) based systems in the context of responsible AI and evolving interface boundaries. It proposes a pattern-oriented reference architecture with three layers (system, operation, supply chain) and seven key design decisions to balance FM capabilities with governance, safety, and maintainability. The architecture is empirically evaluated by mapping to a responsible AI chatbot, incorporating retrieval augmented generation, verifiers, think-aloud explanations, multi-agent orchestration, and an AIBOM registry. The work offers concrete, actionable guidance for building adaptable, responsible FM-based systems and points toward a pattern catalogue to expand design coverage over time.

Abstract

The release of ChatGPT, Gemini, and other large language model has drawn huge interests on foundations models. There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems. However, there is a lack of systematic guidance on the architecture design. Particularly, the the rapidly growing capabilities of foundations models can eventually absorb other components of AI systems, posing challenges of moving boundary and interface evolution in architecture design. Furthermore, incorporating foundations models into AI systems raises significant concerns about responsible and safe AI due to their opaque nature and rapidly advancing intelligence. To address these challenges, the paper first presents an architecture evolution of AI systems in the era of foundation models, transitioning from "foundation-model-as-a-connector" to "foundation-model-as-a-monolithic architecture". The paper then identifies key design decisions and proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems. The patterns can enable the potential of foundation models while ensuring associated risks.
Paper Structure (16 sections, 2 figures)

This paper contains 16 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Architecture evolution of AI systems
  3. Architectural Design decisions
  4. Design decision 1: Different design options for using FMs
  5. Design decision 2: Chain of FMs vs. an ultra-large FM
  6. Design decision 3: Responsibilities of external components
  7. Design decision 4: Automatic response vs. verifier
  8. Design decision 5: Passive interaction vs. proactive interaction
  9. Design decision 6: Single agent vs. multi-agents
  10. Design decision 7: Think aloud vs. think silently
  11. Reference architecture
  12. System layer
  13. Operation layer
  14. Supply chain layer
  15. Evaluation
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Architecture evolution: from "foundation-model-as-a-connector" to "foundation-model-as-a-monolithic-architecture".
  • Figure 2: A pattern-oriented reference architecture for designing responsible FM-based systems.