Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems

Yuhao Wu, Franziska Roesner, Tadayoshi Kohno, Ning Zhang, Umar Iqbal

TL;DR

<3-5 sentence high-level summary> IsolateGPT tackles the security and privacy vulnerabilities of LLM-based agentic systems that run third-party apps by introducing a hub-and-spoke architecture with strict execution isolation. Each app is paired with a dedicated LLM and executes within an isolated spoke, while a central hub coordinates query planning, memory, and inter-spoke collaboration through a controlled inter-spoke communication protocol. The approach preserves the full functionality of non-isolated baselines and demonstrates protection against a wide range of attacks, with measured overheads of under 30% for the majority of queries. The work provides practical architecture, a working prototype, and open-source resources to guide future research on access control and enhanced secure collaboration in LLM-based ecosystems.

Abstract

Large language models (LLMs) extended as systems, such as ChatGPT, have begun supporting third-party applications. These LLM apps leverage the de facto natural language-based automated execution paradigm of LLMs: that is, apps and their interactions are defined in natural language, provided access to user data, and allowed to freely interact with each other and the system. These LLM app ecosystems resemble the settings of earlier computing platforms, where there was insufficient isolation between apps and the system. Because third-party apps may not be trustworthy, and exacerbated by the imprecision of natural language interfaces, the current designs pose security and privacy risks for users. In this paper, we evaluate whether these issues can be addressed through execution isolation and what that isolation might look like in the context of LLM-based systems, where there are arbitrary natural language-based interactions between system components, between LLM and apps, and between apps. To that end, we propose IsolateGPT, a design architecture that demonstrates the feasibility of execution isolation and provides a blueprint for implementing isolation, in LLM-based systems. We evaluate IsolateGPT against a number of attacks and demonstrate that it protects against many security, privacy, and safety issues that exist in non-isolated LLM-based systems, without any loss of functionality. The performance overhead incurred by IsolateGPT to improve security is under 30% for three-quarters of tested queries.

IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems

TL;DR

<3-5 sentence high-level summary> IsolateGPT tackles the security and privacy vulnerabilities of LLM-based agentic systems that run third-party apps by introducing a hub-and-spoke architecture with strict execution isolation. Each app is paired with a dedicated LLM and executes within an isolated spoke, while a central hub coordinates query planning, memory, and inter-spoke collaboration through a controlled inter-spoke communication protocol. The approach preserves the full functionality of non-isolated baselines and demonstrates protection against a wide range of attacks, with measured overheads of under 30% for the majority of queries. The work provides practical architecture, a working prototype, and open-source resources to guide future research on access control and enhanced secure collaboration in LLM-based ecosystems.

Abstract

Large language models (LLMs) extended as systems, such as ChatGPT, have begun supporting third-party applications. These LLM apps leverage the de facto natural language-based automated execution paradigm of LLMs: that is, apps and their interactions are defined in natural language, provided access to user data, and allowed to freely interact with each other and the system. These LLM app ecosystems resemble the settings of earlier computing platforms, where there was insufficient isolation between apps and the system. Because third-party apps may not be trustworthy, and exacerbated by the imprecision of natural language interfaces, the current designs pose security and privacy risks for users. In this paper, we evaluate whether these issues can be addressed through execution isolation and what that isolation might look like in the context of LLM-based systems, where there are arbitrary natural language-based interactions between system components, between LLM and apps, and between apps. To that end, we propose IsolateGPT, a design architecture that demonstrates the feasibility of execution isolation and provides a blueprint for implementing isolation, in LLM-based systems. We evaluate IsolateGPT against a number of attacks and demonstrate that it protects against many security, privacy, and safety issues that exist in non-isolated LLM-based systems, without any loss of functionality. The performance overhead incurred by IsolateGPT to improve security is under 30% for three-quarters of tested queries.
Paper Structure (76 sections, 13 figures, 4 tables)

This paper contains 76 sections, 13 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. LLM-based systems
  4. LLM apps architecture
  5. Security and privacy risks
  6. Securing LLM-based systems
  7. Threat model
  8. System model
  9. Attacker capabilities and goals
  10. Trust relationships
  11. Our scope
  12. In scope
  13. Out of scope
  14. IsolateGPT: System architecture
  15. Hub goals and design
  16. ...and 61 more sections

Figures (13)

  • Figure 1: Query resolution with apps in LLM-based systems: LLM apps (i.e., functionality descriptions & APIs) are loaded in system memory. For each query, the LLM leverages available apps and memory, to generate a step-by-step plan to resolve the query. Based on its plan, the LLM can directly call and exchange information between APIs of needed apps.
  • Figure 2: IsolateGPT's architecture in action: (1) User request to send an email with an attachment from a cloud drive directly goes to the hub operator. (2) Operator consults hub's planner and memory module, to decide app(s) and essential data needed to resolve the query. Based on the plan, the hub operator invokes a spoke with the email app. (3) Email spoke then generates its step-by-step query resolution plan by consulting its LLM and memory module. Since the email spoke needs to collaborate with the cloud drive app, its operator leverages the ISC protocol to establish that connection via the hub with user permission. (4) After query resolution, spoke operator returns the response to the hub operator, which then shows it to user. The hub and spoke operators (colored in green) are non-LLM modules that allow to deterministically exchange well-defined messages between spokes and hub.
  • Figure 3: Collaboration between spokes through ISC protocol. (1) Spoke operator requests the hub operator for a functionality. (2) Hub operator responds by providing the formats in which a request can be sent and a response can be expected. (3) Spoke operator then initiates a request, (4) which the hub operator relays to requested spoke. (5) Spoke then resolves the request and sends a response to the hub operator, (6) which relays it to the calling spoke. Steps 1, 3, and 5 require user consent.
  • Figure 4: Example user permission dialog. It includes hub's assessment of whether a request is unexpected.
  • Figure 5: Summarized execution of two ride sharing apps (one malicious and one benign). The malicious app (Quick Ride) is successfully able to alter the behavior of the benign app (Metro Hail) in VanillaGPT but fails to do so in IsolateGPT.
  • ...and 8 more figures