Optimizing Data Transfer Performance and Energy Efficiency with Deep Reinforcement Learning

TL;DR

This work tackles the challenge of high-throughput, energy-efficient data transfers over shared networks. It introduces SPARTA, a multi-parameter DRL framework that tunes concurrency $cc$ and parallelism $p$ using reward signals that balance throughput, energy, and fairness, and it can pause/resume transfers to adapt to traffic conditions. An emulated training environment accelerates learning by reusing real-world transition logs, enabling rapid convergence across multiple DRL algorithms. Empirical results show up to 25% throughput gains and up to 40% end-system energy reductions relative to baselines, with SPARTA-FE achieving stronger fairness and SPARTA-T delivering favorable throughput-energy trade-offs. The approach demonstrates robust performance across CloudLab, Chameleon, and FABRIC, offering practical benefits for sustainable, high-performance data movement in shared network settings and suggesting pathways for scaling to larger, multi-agent deployments and additional transport protocols.

Abstract

The rapid growth of data across fields of science and industry has increased the need to improve the performance of end-to-end data transfers while using the resources more efficiently. In this paper, we present a dynamic, multiparameter reinforcement learning (RL) framework that adjusts application-layer transfer settings during data transfers on shared networks. Our method strikes a balance between high throughput and low energy utilization by employing reward signals that focus on both energy efficiency and fairness. The RL agents can pause and resume transfer threads as needed, pausing during heavy network use and resuming when resources are available, to prevent overload and save energy. We evaluate several RL techniques and compare our solution with state-of-the-art methods by measuring computational overhead, adaptability, throughput, and energy consumption. Our experiments show up to 25% increase in throughput and up to 40% reduction in energy usage at the end systems compared to baseline methods, highlighting a fair and energy-efficient way to optimize data transfers in shared network environments.

Figures (7)

• Figure 1: The plots show the throughput and energy consumption for different concurrency and parallelism settings under varying background traffic conditions observed at different times of the day in the Chameleon Cloud between TACC and UC sites, connected by a 10 Gbps network link. Subfigure (a) highlights the throughput behavior, while subfigure (b) shows the average energy consumption per measurement interval (MI) of 1 second $(power)$, both influenced by the chosen levels of concurrency and parallelism. These results demonstrate the performance and energy efficiency trade-offs under different background traffic.
• Figure 2: Overview of the offline-online training process, which involves a real environment where data is collected from the source (Src) through the network (Net) to the destination (Dst). The collected data undergoes exploration and logging, followed by clustering using k-means. The resulting data is used for training DRL agents in an emulated environment, with a feedback loop for validation and re-training of the policy.
• Figure 3: Diagram illustrates the primary DRL components for optimizing transfer parameters. The agent observes the state features (e.g., $plr, rtt\_gradient, rtt\_ratio, cc, p$), takes an action ($\{\Delta cc, \Delta p\}$), and receives a reward based on fairness or energy-efficiency objectives, all within an environment.
• Figure 4: Comparison of Various DRL Algorithms trained on offline historical data using F&E Reward (Fairness and Efficiency Reward) and T/E Reward (Throughput-Focused Energy Efficiency Reward), and evaluated in simulation and real-world transfers between Chameleon Cloud (TACC) and UC sites.
• Figure 5: Cumulative reward progression over 500 episodes for agents initially trained on Chameleon Cloud nodes with throughput focused energy objective (T/E) reward signal and subsequently tuned on Cloudlab nodes. The plot illustrates the improvement in cumulative rewards per episode during the tuning process, with each line representing a different agent and shaded areas indicating the variability of the achieved rewards.