Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy and Controllable Professional Knowledge Utilization in Large Language Models with TEE-GPU Execution

Yifeng Cai, Zhida An, Yuhan Meng, Houqian Liu, Pengli Wang, Hanwen Lei, Yao Guo, Ding Li

TL;DR

PKUS addresses the trustworthy and controllable use of high-value professional knowledge in LLM services by separating provider knowledge as compact adapters that run inside attested TEEs while keeping the backbone model on GPUs. The system introduces EdgePrune for compact adapters, AegisProto for hardware-rooted lifecycle management, AlignAgg for structure-aware multi-provider aggregation, and SwiftSched to overlap GPU and enclave computation, achieving scalable, low-latency inference. Empirical results show PKUS preserves near-baseline utility on SST-2, MNLI, and SQuAD across GPT-2 Large and Llama-3.2-1B, with significant speedups over CPU-TEE and cryptographic baselines. The work demonstrates that transparency and separation enable practical, trustworthy, and controllable use of professional knowledge in real-world enterprise LLM deployments, facilitating fairer data-provider economics and safer knowledge sharing.

Abstract

Future improvements in large language model (LLM) services increasingly hinge on access to high-value professional knowledge rather than more generic web data. However, the data providers of this knowledge face a skewed tradeoff between income and risk: they receive little share of downstream value yet retain copyright and privacy liability, making them reluctant to contribute their assets to LLM services. Existing techniques do not offer a trustworthy and controllable way to use professional knowledge, because they keep providers in the dark and combine knowledge parameters with the underlying LLM backbone. In this paper, we present PKUS, the Professional Knowledge Utilization System, which treats professional knowledge as a first-class, separable artifact. PKUS keeps the backbone model on GPUs and encodes each provider's contribution as a compact adapter that executes only inside an attested Trusted Execution Environment (TEE). A hardware-rooted lifecycle protocol, adapter pruning, multi-provider aggregation, and split-execution scheduling together make this design practical at serving time. On SST-2, MNLI, and SQuAD with GPT-2 Large and Llama-3.2-1B, PKUS preserves model utility, matching the accuracy and F1 of full fine-tuning and plain LoRA, while achieving the lowest per-request latency with 8.1-11.9x speedup over CPU-only TEE inference and naive CPU-GPU co-execution.

Trustworthy and Controllable Professional Knowledge Utilization in Large Language Models with TEE-GPU Execution

TL;DR

PKUS addresses the trustworthy and controllable use of high-value professional knowledge in LLM services by separating provider knowledge as compact adapters that run inside attested TEEs while keeping the backbone model on GPUs. The system introduces EdgePrune for compact adapters, AegisProto for hardware-rooted lifecycle management, AlignAgg for structure-aware multi-provider aggregation, and SwiftSched to overlap GPU and enclave computation, achieving scalable, low-latency inference. Empirical results show PKUS preserves near-baseline utility on SST-2, MNLI, and SQuAD across GPT-2 Large and Llama-3.2-1B, with significant speedups over CPU-TEE and cryptographic baselines. The work demonstrates that transparency and separation enable practical, trustworthy, and controllable use of professional knowledge in real-world enterprise LLM deployments, facilitating fairer data-provider economics and safer knowledge sharing.

Abstract

Future improvements in large language model (LLM) services increasingly hinge on access to high-value professional knowledge rather than more generic web data. However, the data providers of this knowledge face a skewed tradeoff between income and risk: they receive little share of downstream value yet retain copyright and privacy liability, making them reluctant to contribute their assets to LLM services. Existing techniques do not offer a trustworthy and controllable way to use professional knowledge, because they keep providers in the dark and combine knowledge parameters with the underlying LLM backbone. In this paper, we present PKUS, the Professional Knowledge Utilization System, which treats professional knowledge as a first-class, separable artifact. PKUS keeps the backbone model on GPUs and encodes each provider's contribution as a compact adapter that executes only inside an attested Trusted Execution Environment (TEE). A hardware-rooted lifecycle protocol, adapter pruning, multi-provider aggregation, and split-execution scheduling together make this design practical at serving time. On SST-2, MNLI, and SQuAD with GPT-2 Large and Llama-3.2-1B, PKUS preserves model utility, matching the accuracy and F1 of full fine-tuning and plain LoRA, while achieving the lowest per-request latency with 8.1-11.9x speedup over CPU-only TEE inference and naive CPU-GPU co-execution.

Paper Structure

This paper contains 17 sections, 2 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Professional Knowledge in LLM Services
  4. Limitations of Existing Approaches
  5. Motivation
  6. Overview
  7. Design
  8. Abstraction and Execution Model
  9. Design of EdgePrune
  10. Design of AegisProto
  11. Design of AlignAgg
  12. Design of SwiftSched
  13. Evaluation
  14. Experiment Configurations
  15. RQ1: Utility
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of PKUS
  • Figure 2: Breakdown of per-request latency into communication, enclave computation, and GPU time.
  • Figure 3: Effectiveness of AlignAgg as the number of data providers $k$ increases. Results are averaged over all $\binom{4}{k}$ subsets. The metric is accuracy (%) for SST-2 and MNLI, and token-level F1 (%) for SQuAD.