Preparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca

TL;DR

Seneca addresses the data preprocessing bottleneck in concurrent multimedia ML training by jointly optimizing cache partitioning across encoded, decoded, and augmented data forms and by sampling cached data opportunistically to improve cache hit rates. It introduces Model-Driven Partitioning (MDP), a high-level DSI throughput model, and Opportunistic Data Sampling (ODS), a cache-aware sampler; implemented by modifying PyTorch's dataloader and Redis caching. In extensive experiments across multiple hardware platforms, models, and datasets, Seneca achieves up to 3.45x DSI throughput improvement and a 45.23% reduction in makespan for concurrent jobs, while preserving model accuracy. The work offers a practical, open-source solution and a generalizable modeling framework for preprocessing-heavy ML pipelines.

Abstract

Input data preprocessing is a common bottleneck when concurrently training multimedia machine learning (ML) models in modern systems. To alleviate these bottlenecks and reduce the training time for concurrent jobs, we present Seneca, a data loading system that optimizes cache partitioning and data sampling for the data storage and ingestion (DSI) pipeline. The design of Seneca contains two key techniques. First, Seneca uses a performance model for the data pipeline to optimally partition the cache for three different forms of data (encoded, decoded, and augmented). Second, Seneca opportunistically serves cached data over uncached ones during random batch sampling so that concurrent jobs benefit from each other. We implement Seneca by modifying PyTorch and demonstrate its effectiveness by comparing it against several state-of-the-art caching systems for DNN training. Seneca reduces the makespan by 45.23% compared to PyTorch and increases data processing throughput by up to 3.45x compared to the next best dataloader.

Figures (15)

• Figure 1: The growing gap between CPU and GPU peak performance in TFLOPS from 2011--2023 awsp2instancetypeawsp3instancetypeAzureNCv4gcpnvidiak20nvidiak40nvidiak80nvidiap100nvidiav100nvidiaa100nvidiah100 (Figure \ref{['fig:gpu_v_cpu_flops']}) resulting in DSI pipeline bottlenecks for multimedia model training (Figure \ref{['fig:gpu_v_cpu_flops_throughput']}).
• Figure 2: The ML training process: (1) fetches data from storage, (2) decodes, transforms, and forms a minibatch, and (3) loads the minibatch into the GPU for training. This repeats until the target accuracy is reached. Encoded data is small, but decoded and augmented data are much larger.
• Figure 3: Fetch, preprocess, and compute times when data is cached in encoded ('E') or augmented ('A') form for ResNet-18 (RN18), ResNet-152 (RN152), VGG-19 (V19), SwinT big (STb), and ViT huge (VTh) with different cache sizes.
• Figure 4: Drawback of OS page-cache for random access patterns (Figure \ref{['fig:dataloader_throughput_memory']}) and the impact of sharing preprocessed data with concurrently training jobs (Figure \ref{['fig:dataloader_throughput_multijob']}).
• Figure 5: The DSI pipeline model.