Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity

Shivnath Tathe

Abstract

Low-precision neural network training has emerged as a promising direction for reducing computational costs and democratizing access to deep learning research. However, existing 4-bit quantization methods either rely on expensive GPU infrastructure or suffer from significant accuracy degradation. In this work, we present a practical method for training convolutional neural networks at true 4-bit precision using standard PyTorch operations on commodity CPUs. We introduce a novel tanh-based soft weight clipping technique that, combined with symmetric quantization, dynamic per-layer scaling, and straight-through estimators, achieves stable convergence and competitive accuracy. Training a VGG-style architecture with 3.25 million parameters from scratch on CIFAR-10, our method achieves 92.34% test accuracy on Google Colab's free CPU tier -- matching full-precision baseline performance (92.5%) with only a 0.16% gap. We further validate on CIFAR-100, achieving 70.94% test accuracy across 100 classes with the same architecture and training procedure, demonstrating that 4-bit training from scratch generalizes to harder classification tasks. Both experiments achieve 8x memory compression over FP32 while maintaining exactly 15 unique weight values per layer throughout training. We additionally validate hardware independence by demonstrating rapid convergence on a consumer mobile device (OnePlus 9R), achieving 83.16% accuracy in only 6 epochs. To the best of our knowledge, no prior work has demonstrated 4-bit quantization-aware training achieving full-precision parity on standard CPU hardware without specialized kernels or post-training quantization.

True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity

Abstract

Low-precision neural network training has emerged as a promising direction for reducing computational costs and democratizing access to deep learning research. However, existing 4-bit quantization methods either rely on expensive GPU infrastructure or suffer from significant accuracy degradation. In this work, we present a practical method for training convolutional neural networks at true 4-bit precision using standard PyTorch operations on commodity CPUs. We introduce a novel tanh-based soft weight clipping technique that, combined with symmetric quantization, dynamic per-layer scaling, and straight-through estimators, achieves stable convergence and competitive accuracy. Training a VGG-style architecture with 3.25 million parameters from scratch on CIFAR-10, our method achieves 92.34% test accuracy on Google Colab's free CPU tier -- matching full-precision baseline performance (92.5%) with only a 0.16% gap. We further validate on CIFAR-100, achieving 70.94% test accuracy across 100 classes with the same architecture and training procedure, demonstrating that 4-bit training from scratch generalizes to harder classification tasks. Both experiments achieve 8x memory compression over FP32 while maintaining exactly 15 unique weight values per layer throughout training. We additionally validate hardware independence by demonstrating rapid convergence on a consumer mobile device (OnePlus 9R), achieving 83.16% accuracy in only 6 epochs. To the best of our knowledge, no prior work has demonstrated 4-bit quantization-aware training achieving full-precision parity on standard CPU hardware without specialized kernels or post-training quantization.
Paper Structure (37 sections, 9 equations, 4 figures, 7 tables)

This paper contains 37 sections, 9 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Contributions
  4. Related Work
  5. Neural Network Quantization
  6. Ultra-Low Precision Training
  7. On-Device Training
  8. Gap in Existing Work
  9. Method
  10. Symmetric 4-Bit Quantization
  11. Dynamic Per-Layer Scaling
  12. Straight-Through Estimator (STE)
  13. Tanh-Based Soft Weight Clipping
  14. Model Architecture
  15. Training Procedure
  16. ...and 22 more sections

Figures (4)

  • Figure 1: Training pipeline for 4-bit quantization-aware training. Tanh soft clipping (highlighted) is applied after each optimizer update, constraining weights before quantization. The loop repeats each training step.
  • Figure 2: Comparison of clipping methods. Tanh soft clipping (our method, blue) provides smooth gradient flow compared to hard clipping (red), which creates gradient discontinuities at boundaries ($\pm 3$).
  • Figure 3: 4-bit training convergence on CIFAR-10. Our method achieves 92.34% test accuracy (epoch 110), matching the FP32 baseline (92.5%, red dashed line) with only 0.16% gap.
  • Figure 4: 4-bit QAT training convergence on CIFAR-100. Best test accuracy of 70.94% is achieved at epoch 104 (green dashed line). The shaded region marks accuracy degradation caused by a learning rate warm restart after epoch 104 -- quantization remained stable (15/15 unique values) throughout, confirming the degradation is scheduler-induced, not quantization-induced.