GPTA: Generative Prompt Tuning Assistant for Synergistic Downstream Neural Network Enhancement with LLMs

TL;DR

GPTA tackles the challenge of efficiently leveraging API-based LLMs for improving downstream task models without exposing private data or incurring excessive costs. It treats the LLM as a teaching assistant that dynamically generates prefix prompts from dataset descriptions, which are prepended to inputs during training. A novel dialogue-gradient mechanism jointly optimizes the LLM and the downstream model, enabling adaptive knowledge alignment with the task domain while keeping the LLM out of inference. Empirical results across six NLP benchmarks show GPTA improves performance, reduces overfitting in low-resource settings, and exhibits transferability of domain-aligned prefixes, highlighting a practical, privacy-preserving path for LLM-assisted training.

Abstract

This study introduces GPTA, a Large Language Model assistance training framework, that enhances the training of downstream task models via prefix prompt. By minimizing data exposure to LLM, the framework addresses the security and legal challenges of applying LLM in downstream task model training. GPTA utilizes a new synergistic training approach, optimizing the downstream models with parameter gradients and LLMs with the novel ``dialogue gradient''. The framework not only demonstrates significant improvements in model performance across six NLP benchmark datasets, but also reduces overfitting in low-resource scenarios effectively. The detailed analyses further validate that our pioneer framework provides a cost-efficient and adaptive method for downstream task model training with LLM support.

Figures (6)

• Figure 1: Forward and backward process of GPTA. Forward process (solid arrows) involves the LLM generating prefix prompts for downstream model input enhancement. The backward process (dotted arrows) uses gradient descent for downstream optimization, then applies "dialogue gradient" for the LLM. Colored texts indicate variables have gradient.
• Figure 2: Prefix Prompt History Collection and Dialogue Gradient Computation
• Figure 3: The performance evaluation of GPTA on low-resource training setting over epochs.
• Figure 4: The training accuracy of LLM finding the next prefix prompt enhancing the downstream task model performance. The performance at step 0 is the performance of the gpt-3.5turbo-0613.
• Figure 5: The performance evaluation of GPTA on low-resource training setting over epochs on all datasets.