Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maxing Out the SVM: Performance Impact of Memory and Program Cache Sizes in the Agave Validator

Turan Vural, Yuki Yuminaga, Alex Petrosyan, Ben Livshits

TL;DR

The paper addresses RAM and program cache bottlenecks in Solana's Agave validator under mainnet-like conditions. It employs controlled experiments that vary RAM from 128 GB to 1.5 TB and test multiple program-cache configurations to quantify throughput, memory behavior, and cache latency. The findings identify ~512 GB as a safe baseline for reliable operation, with 1 TB–1.5 TB offering smoother performance and faster catch-up, while 128 GB is non-viable. It also shows that enlarging the program cache to 1024–2048 entries markedly reduces program-cache latency (up to ~90% reduction) and evictions, providing clear guidance for hardware provisioning and caching strategies to improve validator efficiency.

Abstract

In this paper we analyze some of the bottlenecks in the execution pipeline of Solana's Agave validator client, focusing on RAM and program cache usage under mainnet conditions. Through a series of controlled experiments, we measure the validator's throughput and resource efficiency as RAM availability ranges between 128 GB to 1,536 GB (1.5 TB). We discover that the validator performance degrades significantly below 256 GB, with transaction processing falling behind real-time block production. Additionally, we study the program cache behavior, identifying inefficiencies in program eviction and load latency. Our results provide practical guidance for hardware provisioning and suggest improvements to the Solana execution and caching strategy, reducing latency due to the program cache by 90%.

Maxing Out the SVM: Performance Impact of Memory and Program Cache Sizes in the Agave Validator

TL;DR

The paper addresses RAM and program cache bottlenecks in Solana's Agave validator under mainnet-like conditions. It employs controlled experiments that vary RAM from 128 GB to 1.5 TB and test multiple program-cache configurations to quantify throughput, memory behavior, and cache latency. The findings identify ~512 GB as a safe baseline for reliable operation, with 1 TB–1.5 TB offering smoother performance and faster catch-up, while 128 GB is non-viable. It also shows that enlarging the program cache to 1024–2048 entries markedly reduces program-cache latency (up to ~90% reduction) and evictions, providing clear guidance for hardware provisioning and caching strategies to improve validator efficiency.

Abstract

In this paper we analyze some of the bottlenecks in the execution pipeline of Solana's Agave validator client, focusing on RAM and program cache usage under mainnet conditions. Through a series of controlled experiments, we measure the validator's throughput and resource efficiency as RAM availability ranges between 128 GB to 1,536 GB (1.5 TB). We discover that the validator performance degrades significantly below 256 GB, with transaction processing falling behind real-time block production. Additionally, we study the program cache behavior, identifying inefficiencies in program eviction and load latency. Our results provide practical guidance for hardware provisioning and suggest improvements to the Solana execution and caching strategy, reducing latency due to the program cache by 90%.
Paper Structure (18 sections, 13 figures)

This paper contains 18 sections, 13 figures.

Table of Contents

  1. Introduction
  2. Paper Organization
  3. Background
  4. Program Cache
  5. Execution
  6. Methodology
  7. Results
  8. TPS Performance and Catch-up Behavior
  9. Memory Utilization and Page Fault Behavior
  10. Discussion
  11. Program Cache
  12. Methodology
  13. Metrics Measured
  14. Memory Analysis
  15. Cache and TPS Analysis
  16. ...and 3 more sections

Figures (13)

  • Figure 1: The CPU configuration of the test machine
  • Figure 2: Drive layout and performance characteristics for the test machine.
  • Figure 3: Validator performance across RAM configurations and test runs. The 0 TPS time is the time in which the validator had reported zero transactions processed.
  • Figure 4: Comparison of TPS instances across validators with varying memory sizes. 1.5 TB shows the most instances of TPS corresponding to the average TPS of the validator across the experiment. 1 TB shows similar behavior with a skew towards slightly above higher instances. 512 GB shows a significant need for catch up, with comparatively more instances of higher TPS and less of average/low TPS, suggesting that it cannot keep up with transaction ingestion. 256 GB shows insufficient instances altogether.
  • Figure 5: Program cache test machine configuration.
  • ...and 8 more figures