Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PoGO: A Scalable Proof of Useful Work via Quantized Gradient Descent and Merkle Proofs

José I. Orlicki

TL;DR

PoGO tackles the challenge of turning expensive ML training into verifiable blockchain work by requiring miners to demonstrate genuine progress on a loss function $\mathcal{L}(\theta)$. It combines 4-bit quantized gradients with a 32-bit Merkle-tree commitment and a two-phase randomness-based random leaf verification to enable scalable, cryptographic proofs of training progress. An empirical cost analysis shows verification is orders of magnitude cheaper than training for enormous models such as GPT-3 and Gemma-3, justifying the feasibility of a wide-area, incentive-compatible PoUW protocol, albeit with longer block times to accommodate meaningful learning. The framework also addresses data availability, attestation-based finalization, incentive structures, and potential extensions like fine-tuning and zero-knowledge enhancements, highlighting a path toward a robust decentralized ML training ecosystem.

Abstract

We present a design called Proof of Gradient Optimization (PoGO) for blockchain consensus, where miners produce verifiable evidence of training large-scale machine-learning models. Building on previous work, we incorporate quantized gradients (4-bit precision) to reduce storage and computation requirements, while still preserving the ability of verifiers to check that real progress has been made on lowering the model's loss. Additionally, we employ Merkle proofs over the full 32-bit model to handle large parameter sets and to enable random leaf checks with minimal on-chain data. We illustrate these ideas using GPT-3 (175B parameters) as a reference example and also refer to smaller but high-performance models (e.g., Gemma~3 with 27B parameters). We provide an empirical cost analysis showing that verification is significantly cheaper than training, thanks in part to quantization and sampling. We also discuss the necessity of longer block times (potentially hours) when incorporating meaningful training steps, the trade-offs when using specialized GPU hardware, and how binary diffs may incrementally optimize updates. Finally, we note that fine-tuning can be handled in a similar manner, merely changing the dataset and the manner of sampling but preserving the overall verification flow. Our protocol allows verifiers to issue either positive or negative attestations; these are aggregated at finalization to either confirm the update or slash the miner.

PoGO: A Scalable Proof of Useful Work via Quantized Gradient Descent and Merkle Proofs

TL;DR

PoGO tackles the challenge of turning expensive ML training into verifiable blockchain work by requiring miners to demonstrate genuine progress on a loss function . It combines 4-bit quantized gradients with a 32-bit Merkle-tree commitment and a two-phase randomness-based random leaf verification to enable scalable, cryptographic proofs of training progress. An empirical cost analysis shows verification is orders of magnitude cheaper than training for enormous models such as GPT-3 and Gemma-3, justifying the feasibility of a wide-area, incentive-compatible PoUW protocol, albeit with longer block times to accommodate meaningful learning. The framework also addresses data availability, attestation-based finalization, incentive structures, and potential extensions like fine-tuning and zero-knowledge enhancements, highlighting a path toward a robust decentralized ML training ecosystem.

Abstract

We present a design called Proof of Gradient Optimization (PoGO) for blockchain consensus, where miners produce verifiable evidence of training large-scale machine-learning models. Building on previous work, we incorporate quantized gradients (4-bit precision) to reduce storage and computation requirements, while still preserving the ability of verifiers to check that real progress has been made on lowering the model's loss. Additionally, we employ Merkle proofs over the full 32-bit model to handle large parameter sets and to enable random leaf checks with minimal on-chain data. We illustrate these ideas using GPT-3 (175B parameters) as a reference example and also refer to smaller but high-performance models (e.g., Gemma~3 with 27B parameters). We provide an empirical cost analysis showing that verification is significantly cheaper than training, thanks in part to quantization and sampling. We also discuss the necessity of longer block times (potentially hours) when incorporating meaningful training steps, the trade-offs when using specialized GPU hardware, and how binary diffs may incrementally optimize updates. Finally, we note that fine-tuning can be handled in a similar manner, merely changing the dataset and the manner of sampling but preserving the overall verification flow. Our protocol allows verifiers to issue either positive or negative attestations; these are aggregated at finalization to either confirm the update or slash the miner.

Paper Structure

This paper contains 48 sections, 12 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Key contributions.
  3. Background and Motivation
  4. Large-Scale ML Models on Blockchain
  5. Why Quantization?
  6. Protocol Overview
  7. Data Structures and Merkle Proofs
  8. Merkle Tree for the 32-bit Model
  9. Random Leaf Reveal
  10. Quantized vs. Full-Precision Loss Checking
  11. Training in 32-bit, Verifying in 4-bit
  12. Why This Works
  13. Empirical Cost Estimates: GPT-3 and Gemma 3
  14. Model Sizes in Bytes
  15. Ratio of Computation: Miners vs. Verifiers
  16. ...and 33 more sections

Figures (1)

  • Figure 1: Timeline of PoGO v2. The miner commits to the new full model and its quantized version at block N, publishes the 4-bit model by block $N + w/2$, and must reveal a random Merkle leaf (32-bit data) at block $N + w/2$. Verifiers issue positive or negative attestations, which are aggregated at block $N + w$ to finalize or slash.