Encapsulating Knowledge in One Prompt

TL;DR

KiOP introduces Knowledge in One Prompt, a data-free paradigm that encapsulates knowledge from multiple frozen models into a single Visual Prompt. By partitioning the prompt into a Prompt Core and Prompt Periphery and employing a Synthesize System with data banks, KiOP enables parallel, storage-efficient knowledge transfer without real training data. The method demonstrates strong performance across diverse dataset–backbone pairs and remains effective in cross-architecture and multi-model scenarios, while dramatically reducing trainable parameters. This approach advances practical data-free transfer and serving of concurrent knowledge-transfer requests in realistic settings. KiOP achieves competitive or superior knowledge encapsulation with substantially lower storage and preserves source-model knowledge, offering a scalable solution for parallel, data-free knowledge transfer.

Abstract

This paradigm encapsulates knowledge from various models into a solitary prompt without altering the original models or requiring access to the training data, which enables us to achieve efficient and convenient knowledge transfer in more realistic scenarios. From a practicality standpoint, this paradigm not only for the first time proves the effectiveness of Visual Prompt in data inaccessible contexts, but also solves the problems of low model reusability and high storage resource consumption faced by traditional Data-Free Knowledge Transfer, which means that we can realize the parallel knowledge transfer of multiple models without modifying any source model. Extensive experiments across various datasets and models demonstrate the efficacy of the proposed KiOP knowledge transfer paradigm. Without access to real training data and with rigorous storage capacity constraints, it is also capable of yielding considerable outcomes when dealing with cross-model backbone setups and handling parallel knowledge transfer processing requests with multiple (more than 2) models.

Figures (6)

• Figure 1: The schematic illustrates Visual Prompt Learning, Conventional Data-Free Knowledge Distillation (DFKD), and our novel Data-free Knowledge Transfer Framework (KiOP). Visual Prompt Learning adjusts pre-trained models for various downstream tasks by appending prompts to inputs while leaving the model frozen. DFKD facilitates knowledge transfer from a teacher model to a student model in the absence of real data. Our KiOP framework ensures effective knowledge transfer without modifications to the pre-trained model or access to real data.
• Figure 2: A comprehensive illustration of the proposed paradigm: (a) Overall pipeline, (b) Model fusion and input logic, (c) Synthetic data generation.
• Figure 3: $\mathcal{VP}$ and its corresponding Grad-Cam when (Dts.$\mathcal{A}$, Dts.$\mathcal{B}$) = (cifar10, svhn), ResNet-18 as backbone. Size of $\mathcal{PC}$ is set to 64 and $\mathcal{PP}$ is 128.
• Figure 4: VP manages the knowledge storage for three distinct models simultaneously.
• Figure 5: KiOP achieves low storage resource usage.