Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proof of Quality: A Costless Paradigm for Trustless Generative AI Model Inference on Blockchains

Zhenjie Zhang, Yuyang Rao, Hao Xiao, Xiaokui Xiao, Yin Yang

TL;DR

This work introduces Proof of Quality (PoQ), a paradigm for trustless generative AI inference on blockchains that prioritizes output quality over the inference process itself. It proposes PQML, a practical NLP-focused protocol that uses lightweight cross-encoders for quality assessment and a two-phase encrypted consensus to determine rewards, achieving low overhead and rapid consensus. The paper analyzes adversarial robustness and presents optimization strategies (fast consensus, deterministic node selection) alongside empirical studies demonstrating effectiveness and efficiency in NLP workloads. The approach enables scalable, decentralized, and verifiable AI services with minimal computational burden on validators, potentially enabling broader blockchain deployment of large-gen AI models. All math and key equations are integrated to formalize reward and scoring mechanisms, ensuring rigorous incentives and robust behavior in decentralized settings.

Abstract

Generative AI models, such as GPT-4 and Stable Diffusion, have demonstrated powerful and disruptive capabilities in natural language and image tasks. However, deploying these models in decentralized environments remains challenging. Unlike traditional centralized deployment, systematically guaranteeing the integrity of AI model services in fully decentralized environments, particularly on trustless blockchains, is both crucial and difficult. In this paper, we present a new inference paradigm called \emph{proof of quality} (PoQ) to enable the deployment of arbitrarily large generative models on blockchain architecture. Unlike traditional approaches based on validating inference procedures, such as ZKML or OPML, our PoQ paradigm focuses on the outcome quality of model inference. Using lightweight BERT-based cross-encoders as our underlying quality evaluation model, we design and implement PQML, the first practical protocol for real-world NLP generative model inference on blockchains, tailored for popular open-source models such as Llama 3 and Mixtral. Our analysis demonstrates that our protocol is robust against adversarial but rational participants in ecosystems, where lazy or dishonest behavior results in fewer benefits compared to well-behaving participants. The computational overhead of validating the quality evaluation is minimal, allowing quality validators to complete the quality check within a second, even using only a CPU. Preliminary simulation results show that PoQ consensus is generated in milliseconds, 1,000 times faster than any existing scheme.

Proof of Quality: A Costless Paradigm for Trustless Generative AI Model Inference on Blockchains

TL;DR

This work introduces Proof of Quality (PoQ), a paradigm for trustless generative AI inference on blockchains that prioritizes output quality over the inference process itself. It proposes PQML, a practical NLP-focused protocol that uses lightweight cross-encoders for quality assessment and a two-phase encrypted consensus to determine rewards, achieving low overhead and rapid consensus. The paper analyzes adversarial robustness and presents optimization strategies (fast consensus, deterministic node selection) alongside empirical studies demonstrating effectiveness and efficiency in NLP workloads. The approach enables scalable, decentralized, and verifiable AI services with minimal computational burden on validators, potentially enabling broader blockchain deployment of large-gen AI models. All math and key equations are integrated to formalize reward and scoring mechanisms, ensuring rigorous incentives and robust behavior in decentralized settings.

Abstract

Generative AI models, such as GPT-4 and Stable Diffusion, have demonstrated powerful and disruptive capabilities in natural language and image tasks. However, deploying these models in decentralized environments remains challenging. Unlike traditional centralized deployment, systematically guaranteeing the integrity of AI model services in fully decentralized environments, particularly on trustless blockchains, is both crucial and difficult. In this paper, we present a new inference paradigm called \emph{proof of quality} (PoQ) to enable the deployment of arbitrarily large generative models on blockchain architecture. Unlike traditional approaches based on validating inference procedures, such as ZKML or OPML, our PoQ paradigm focuses on the outcome quality of model inference. Using lightweight BERT-based cross-encoders as our underlying quality evaluation model, we design and implement PQML, the first practical protocol for real-world NLP generative model inference on blockchains, tailored for popular open-source models such as Llama 3 and Mixtral. Our analysis demonstrates that our protocol is robust against adversarial but rational participants in ecosystems, where lazy or dishonest behavior results in fewer benefits compared to well-behaving participants. The computational overhead of validating the quality evaluation is minimal, allowing quality validators to complete the quality check within a second, even using only a CPU. Preliminary simulation results show that PoQ consensus is generated in milliseconds, 1,000 times faster than any existing scheme.
Paper Structure (14 sections, 2 theorems, 9 equations, 5 figures, 3 tables)

This paper contains 14 sections, 2 theorems, 9 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proof of Quality
  3. PQML Protocol
  4. Quality Assessment Method
  5. Consensus Mechanism
  6. Incentive Scheme
  7. Adversarial Analysis
  8. Performance Optimization
  9. Fast Consensus
  10. Deterministic Node Selection
  11. Empirical Studies
  12. Effectiveness
  13. Efficiency
  14. Concluding Remarks

Key Result

Theorem 1

Given a list of models $\{F_1,F_2,\ldots,F_n\}$ associated quality-cost matrix $\{(e_1,c_1),(e_2,c_2),\ldots,(e_n,c_n)\}$ in non-ascending order on $c_j$, there exists a threshold $\theta$, when $\alpha\geq\theta$ in the reward distribution function $\chi$, the inference node would always choose the

Figures (5)

  • Figure 1: While standard inference only generates the response $r$ based on specified input query $q$, ZKML and PQML generate auxiliary information, i.e., zero-knowledge proof $p$ and verifiable results $v$ respectively, to enable the user or the verifiers to check the integreity of the whole computation process. PoQ adopts a very different strategy. It relies on third-party quality assessors to check the model output quality by evaluating the query-response pair $(q,r)$.
  • Figure 2: In this example, three assessors are responsible for generating the quality scores, and $k=2$ expected quality scores are needed for quality consensus. Assessor 1 and Assessor 2 publish their encrypted scores in a mutable and shared storage system. When $k=2$ scores are available, successful quality score nodes upload their corresponding public keys. A late arrival of an encrypted score from Assessor 3 is not included in the consensus calculation.
  • Figure 3: In this example, we illustrate a running example with 4 nodes in the inference pool. The orange node is selected for the first query processing because it has the highest energy value at 10. After the query is finished, the energy value of the orange node is reset to 0, while other nodes' energy values are incremented according to the step value. The green node overtakes the blue node because its step value is higher. The orange node's step value gains a 50% bonus because of its good performance in the processing of the first query. After the completion of the second query, the grey node receives the additional bonus on the step value, because it has waited for too long for a concrete task. This grants the grey node a chance to process the third query in the running example.
  • Figure 4: Reward distribution of GPT4 and Mixtral 8x7b under different configurations of $\alpha$.
  • Figure 5: When varying the number of validators $k$ for inference tasks, the overall latency of consensus remains stable at around 50 milliseconds. We also plot the average latency of cross-encoder calculation at around 30 milliseconds.

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof