Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Profiling Apple Silicon Performance for ML Training

Dahua Feng, Zhiming Xu, Rongxiang Wang, Felix Xiaozhu Lin

TL;DR

The paper investigates end-to-end LLM training on Apple Silicon versus NVIDIA GPUs, focusing on unified memory and system-level factors that influence performance. It employs multiple memory scenarios and workloads (e.g., Whisper, GPT-2) to quantify time per training pass, memory behavior, and kernel performance, and analyzes BLAS kernels to explain the observed gaps. Key findings show Apple Silicon underperforms in raw training throughput but excels in energy efficiency, with memory management, page faults, and kernel-launch overhead contributing to the performance gap; MLX and MPS provide mixed results for Apple Silicon’s BLAS workloads. The work offers practical guidance for practitioners, hardware/software vendors, and benchmark developers on when Apple Silicon is viable and where improvements are most needed to close the gap with NVIDIA GPUs.

Abstract

Apple Silicon has attracted much attention for its performance and role in machine learning (ML) training. Unlike NVIDIA GPUs, which have traditionally dominated ML training, Apple Silicon has a significant difference in memory architecture. It uses Unified Memory, which integrates CPU and GPU memory instead of separate CPU memory and GPU VRAM. However, it is difficult to tell whether Unified Memory means more performance benefits. This paper investigates the performance differences by training several large language model (LLM) workloads end-to-end under different memory scenarios. The results show a significant performance gap between Apple Silicon and NVIDIA GPUs. This paper attributes this gap to system-level factors such as page faults, power consumption, and kernel launch time. In addition, the performance difference of basic linear algebra subprograms (BLAS) on the NVIDIA GPUs and Apple Silicon chips is analyzed to further explain the observed gap.

Profiling Apple Silicon Performance for ML Training

TL;DR

The paper investigates end-to-end LLM training on Apple Silicon versus NVIDIA GPUs, focusing on unified memory and system-level factors that influence performance. It employs multiple memory scenarios and workloads (e.g., Whisper, GPT-2) to quantify time per training pass, memory behavior, and kernel performance, and analyzes BLAS kernels to explain the observed gaps. Key findings show Apple Silicon underperforms in raw training throughput but excels in energy efficiency, with memory management, page faults, and kernel-launch overhead contributing to the performance gap; MLX and MPS provide mixed results for Apple Silicon’s BLAS workloads. The work offers practical guidance for practitioners, hardware/software vendors, and benchmark developers on when Apple Silicon is viable and where improvements are most needed to close the gap with NVIDIA GPUs.

Abstract

Apple Silicon has attracted much attention for its performance and role in machine learning (ML) training. Unlike NVIDIA GPUs, which have traditionally dominated ML training, Apple Silicon has a significant difference in memory architecture. It uses Unified Memory, which integrates CPU and GPU memory instead of separate CPU memory and GPU VRAM. However, it is difficult to tell whether Unified Memory means more performance benefits. This paper investigates the performance differences by training several large language model (LLM) workloads end-to-end under different memory scenarios. The results show a significant performance gap between Apple Silicon and NVIDIA GPUs. This paper attributes this gap to system-level factors such as page faults, power consumption, and kernel launch time. In addition, the performance difference of basic linear algebra subprograms (BLAS) on the NVIDIA GPUs and Apple Silicon chips is analyzed to further explain the observed gap.
Paper Structure (24 sections, 7 figures, 6 tables)

This paper contains 24 sections, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Hardware Characteristics
  4. Apple Silicon.
  5. Software Support
  6. Pytorch.
  7. MLX.
  8. Method
  9. Choices of hardware
  10. Methodologies
  11. Findings
  12. End-to-end evaluations
  13. Precision
  14. Energy efficiency
  15. System issues
  16. ...and 9 more sections

Figures (7)

  • Figure 1: The software/hardware stack for ML training, showing for both Apple GPU and NVIDIA GPU.
  • Figure 2: Different scenarios of training.
  • Figure 3: The forward time, backward time on different devices on the two workloads.
  • Figure 4: Memory consumption over time during near-capacity training of Whisper-large on M2 Max.
  • Figure 5: Measurements on Matrix-Matrix product.
  • ...and 2 more figures