Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating Transfer Learning with Near-Data Computation on Cloud Object Stores

Diana Petrescu, Arsany Guirguis, Do Le Quoc, Javier Picorel, Rachid Guerraoui, Florin Dinu

TL;DR

This work proposes HAPI, a new TL processing system centered around two complementary techniques that address challenges introduced by disaggregation, and judiciously splits the TL computation during the feature extraction phase yielding pushdowns that not only improve network time but also improve total TL training time.

Abstract

Storage disaggregation underlies today's cloud and is naturally complemented by pushing down some computation to storage, thus mitigating the potential network bottleneck between the storage and compute tiers. We show how ML training benefits from storage pushdowns by focusing on transfer learning (TL), the widespread technique that democratizes ML by reusing existing knowledge on related tasks. We propose HAPI, a new TL processing system centered around two complementary techniques that address challenges introduced by disaggregation. First, applications must carefully balance execution across tiers for performance. HAPI judiciously splits the TL computation during the feature extraction phase yielding pushdowns that not only improve network time but also improve total TL training time by overlapping the execution of consecutive training iterations across tiers. Second, operators want resource efficiency from the storage-side computational resources. HAPI employs storage-side batch size adaptation allowing increased storage-side pushdown concurrency without affecting training accuracy. HAPI yields up to 2.5x training speed-up while choosing in 86.8% of cases the best performing split point or one that is at most 5% off from the best.

Accelerating Transfer Learning with Near-Data Computation on Cloud Object Stores

TL;DR

This work proposes HAPI, a new TL processing system centered around two complementary techniques that address challenges introduced by disaggregation, and judiciously splits the TL computation during the feature extraction phase yielding pushdowns that not only improve network time but also improve total TL training time.

Abstract

Storage disaggregation underlies today's cloud and is naturally complemented by pushing down some computation to storage, thus mitigating the potential network bottleneck between the storage and compute tiers. We show how ML training benefits from storage pushdowns by focusing on transfer learning (TL), the widespread technique that democratizes ML by reusing existing knowledge on related tasks. We propose HAPI, a new TL processing system centered around two complementary techniques that address challenges introduced by disaggregation. First, applications must carefully balance execution across tiers for performance. HAPI judiciously splits the TL computation during the feature extraction phase yielding pushdowns that not only improve network time but also improve total TL training time by overlapping the execution of consecutive training iterations across tiers. Second, operators want resource efficiency from the storage-side computational resources. HAPI employs storage-side batch size adaptation allowing increased storage-side pushdown concurrency without affecting training accuracy. HAPI yields up to 2.5x training speed-up while choosing in 86.8% of cases the best performing split point or one that is at most 5% off from the best.
Paper Structure (24 sections, 14 figures, 4 tables, 1 algorithm)

This paper contains 24 sections, 14 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Transfer learning
  4. Cloud object stores
  5. Hardware-accelerated pushdowns
  6. Measurement study
  7. Design and Implementation
  8. Main insights
  9. Design overview
  10. Splitting TL between storage and compute tiers
  11. Storage-side batch size adaptation
  12. System implementation
  13. Evaluation
  14. Methodology
  15. Hapi vs not splitting (NoSplit)
  16. ...and 9 more sections

Figures (14)

  • Figure 1: Overview of TL fine-tuning.
  • Figure 2: A measurement study of 3 per-layer properties for 7 popular DNNs.
  • Figure 3: Speed-up when splitting at various layers normalized to splitting at the freezing layer.
  • Figure 4: The high-level architecture of Hapi. The Hapi client resides in the compute tier, and the Hapi server resides in the COS.
  • Figure 5: Hapi request flow. A TL job is split at layer 10 with training batch size 3000. 3 requests are sent in parallel to the Hapi server. On the COS, for each request, feature extraction uses a smaller COS batch size of 200.
  • ...and 9 more figures