Vision Language Model-Empowered Contract Theory for AIGC Task Allocation in Teleoperation

TL;DR

The paper tackles nighttime teleoperation when AIGC-based low-light image enhancement must be offloaded to diffusion models on edge servers. It proposes a VLM-empowered contract theory framework with two components: automatic difficulty evaluation of AIGC tasks via Vision-Language Models and contract-based pricing to address information asymmetry in task difficulty. The approach yields differential pricing and task-offloading decisions that improve teleoperator utility by about $10.88\%$–$12.43\%$ and edge-server utility by about $1.4\%$–$2.17\%$ in simulations, with competitive performance to human-designed contracts. This framework enables automatic, incentive-compatible AIGC offloading for practical teleoperation, reducing manual difficulty labeling and aligning operator experience with edge resources.

Abstract

Integrating low-light image enhancement techniques, in which diffusion-based AI-generated content (AIGC) models are promising, is necessary to enhance nighttime teleoperation. Remarkably, the AIGC model is computation-intensive, thus necessitating the allocation of AIGC tasks to edge servers with ample computational resources. Given the distinct cost of the AIGC model trained with varying-sized datasets and AIGC tasks possessing disparate demand, it is imperative to formulate a differential pricing strategy to optimize the utility of teleoperators and edge servers concurrently. Nonetheless, the pricing strategy formulation is under information asymmetry, i.e., the demand (e.g., the difficulty level of AIGC tasks and their distribution) of AIGC tasks is hidden information to edge servers. Additionally, manually assessing the difficulty level of AIGC tasks is tedious and unnecessary for teleoperators. To this end, we devise a framework of AIGC task allocation assisted by the Vision Language Model (VLM)-empowered contract theory, which includes two components: VLM-empowered difficulty assessment and contract theory-assisted AIGC task allocation. The first component enables automatic and accurate AIGC task difficulty assessment. The second component is capable of formulating the pricing strategy for edge servers under information asymmetry, thereby optimizing the utility of both edge servers and teleoperators. The simulation results demonstrated that our proposed framework can improve the average utility of teleoperators and edge servers by 10.88~12.43% and 1.4~2.17%, respectively. Code and data are available at https://github.com/ZiJun0819/VLM-Contract-Theory.

Figures (10)

• Figure 1: Potential consequences for teleoperators operating in low-light condition. In each item, the left sub-figure depicts a low-light image, while the right sub-figure shows the corresponding normal-light image.
• Figure 2: Results of AIGC tasks by diffusion-based AIGC models that are trained with varying-sized datasets.
• Figure 3: Evaluation score of AIGC tasks on AIGC models trained with varying-sized datasets. The evaluation score is the addition of the LPIPS zhang2018unreasonable and SSIM wang2004image value of processed results.
• Figure 4: Framework overview of AIGC task allocation assisted by VLM-empowered contract theory in teleoperation.
• Figure 5: Flowchart of AIGC task allocation assisted by VLM-empowered contract theory in teleoperation.

Theorems & Definitions (1)

• Proposition 1