Blockchain-based Crowdsourced Deep Reinforcement Learning as a Service

TL;DR

This work tackles the accessibility barrier of deep reinforcement learning by introducing a blockchain-based crowdsourced DRLaaS that supports two task types: DRL training and model sharing. It establishes QoS-based worker recruitment on a Consortium Blockchain with IPFS-stored models, enabling traceable, autonomous task execution and incentives. Key methodological contributions include task/model attribute definitions, environment-aware similarity metrics, greedy recruitment, and three smart contracts (UMC, TMC, MMC) for decentralized coordination. Empirical evaluation on multi-agent DRL scenarios shows that CPU/GPU resources and model similarity materially impact convergence and training speed, demonstrating the framework's potential for scalable, low-friction DRL access in practice.

Abstract

Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for solving complex problems. However, its full potential remains inaccessible to a broader audience due to its complexity, which requires expertise in training and designing DRL solutions, high computational capabilities, and sometimes access to pre-trained models. This necessitates the need for hassle-free services that increase the availability of DRL solutions to a variety of users. To enhance the accessibility to DRL services, this paper proposes a novel blockchain-based crowdsourced DRL as a Service (DRLaaS) framework. The framework provides DRL-related services to users, covering two types of tasks: DRL training and model sharing. Through crowdsourcing, users could benefit from the expertise and computational capabilities of workers to train DRL solutions. Model sharing could help users gain access to pre-trained models, shared by workers in return for incentives, which can help train new DRL solutions using methods in knowledge transfer. The DRLaaS framework is built on top of a Consortium Blockchain to enable traceable and autonomous execution. Smart Contracts are designed to manage worker and model allocation, which are stored using the InterPlanetary File System (IPFS) to ensure tamper-proof data distribution. The framework is tested on several DRL applications, proving its efficacy.

Figures (10)

• Figure 1: A general overview of the proposed framework.
• Figure 2: The proposed Blockchain-assisted DRLaaS framework. The different steps could involve a single entity, or an interaction between two entities. Entities include requesters, workers, the blockchain, and IPFS.
• Figure 3: Flowchart of the recruitment optimization process.
• Figure 4: The interactions between the users and smart contracts as part of the proposed framework.
• Figure 5: Use-case scenarios of the DRL application environments used to validate the proposed methods.