Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management

Zaifeng Pan, Yipeng Shen, Zhengding Hu, Zhuang Wang, Aninda Manocha, Zheng Wang, Zhongkai Yu, Yue Guan, Yufei Ding

TL;DR

ScaleSim tackles GPU memory pressure in large-scale LLM-based multi-agent simulations by exploiting two key workload properties: sparse activation and estimable invocation orders. It introduces invocation distance as a per-agent metric and provides a unified interface for memory management, enabling distance-guided prefetching and future reuse-aware eviction. Evaluations across AgentSociety, Generative Agents, and information diffusion show ScaleSim achieving up to $1.74\times$ speedup over SGLang, with improved Time-to-First-Token and better scalability. The framework offers an extensible agent-specific memory abstraction and demonstrates practical gains for simulation-centric LLM serving.

Abstract

LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-resident states, including models, prefix caches, and adapters, which quickly exhaust device memory as the agent count grows. We identify two key properties of these workloads: sparse agent activation and an estimable agent invocation order. Based on an analysis of representative workload classes, we introduce invocation distance, a unified abstraction that estimates the relative order in which agents will issue future LLM requests. Leveraging this abstraction, we present ScaleSim, a memory-efficient LLM serving system for large-scale multi-agent simulations. ScaleSim enables proactive prefetching and priority-based eviction, supports diverse agent-specific memory through a modular interface, and achieves up to 1.74x speedup over SGLang on simulation benchmarks.

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management

TL;DR

ScaleSim tackles GPU memory pressure in large-scale LLM-based multi-agent simulations by exploiting two key workload properties: sparse activation and estimable invocation orders. It introduces invocation distance as a per-agent metric and provides a unified interface for memory management, enabling distance-guided prefetching and future reuse-aware eviction. Evaluations across AgentSociety, Generative Agents, and information diffusion show ScaleSim achieving up to speedup over SGLang, with improved Time-to-First-Token and better scalability. The framework offers an extensible agent-specific memory abstraction and demonstrates practical gains for simulation-centric LLM serving.

Abstract

LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-resident states, including models, prefix caches, and adapters, which quickly exhaust device memory as the agent count grows. We identify two key properties of these workloads: sparse agent activation and an estimable agent invocation order. Based on an analysis of representative workload classes, we introduce invocation distance, a unified abstraction that estimates the relative order in which agents will issue future LLM requests. Leveraging this abstraction, we present ScaleSim, a memory-efficient LLM serving system for large-scale multi-agent simulations. ScaleSim enables proactive prefetching and priority-based eviction, supports diverse agent-specific memory through a modular interface, and achieves up to 1.74x speedup over SGLang on simulation benchmarks.
Paper Structure (32 sections, 2 equations, 15 figures, 2 tables)

This paper contains 32 sections, 2 equations, 15 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. LLM-based multi-agent simulation.
  4. Agent-specific memory.
  5. LLM serving systems.
  6. Invocation Distance-Based Memory Management
  7. Limitation of Existing Systems
  8. Frequent reactive memory swaps.
  9. Suboptimal eviction decisions.
  10. Invocation Distance Abstraction for Different Simulation Applications
  11. Independent simulation.
  12. Interaction-involved simulation.
  13. Predefined activation paths.
  14. Efficient Memory Management with Unified Interface
  15. Distance-guided proactive prefetch.
  16. ...and 17 more sections

Figures (15)

  • Figure 1: Illustration of multi-agent simulation phases and agent-specific memory consumption.
  • Figure 2: I/O overhead per simulation step increases with the number of simulation agents.
  • Figure 3: Only a small fraction of agents are activated during each simulation step.
  • Figure 4: Inefficiency of existing systems due to frequent reactive memory swaps and suboptimal eviction decisions.
  • Figure 5: Three representative categories of multi-agent simulation workloads. In each case, we define the invocation distances of agents accordingly and pass them to ScaleSim through a unified interface.
  • ...and 10 more figures