Shuttle Between the Instructions and the Parameters of Large Language Models

TL;DR

SHIP introduces a hybrid VAE-VIB framework that learns mutual mappings between task instructions and LLM parameters, enabling from-instructions-to-parameters and from-parameters-to-instructions capabilities. The model encodes instructions into a latent z that can augment the Task LLM, while also reconstructing instructions from z to enable interpretability and manipulation. Through deductive and inductive verification tasks, SHIP outperforms baselines in both directions and, when combining the two mappings, achieves strong inductive reasoning with data-efficient few-shot learning. The approach demonstrates latent-space alignment between semantic instruction representations and learned parameters, offering a principled route to rapid adaptation and explanation of LLM behavior. While effective, its scope is currently instruction-centric, with future work to extend SHIP to broader task modalities and data sources.

Abstract

The interaction with Large Language Models (LLMs) through instructions has been extensively investigated in the research community. While instructions have been widely used as the guidelines for task solving, this paper further notices that both instructions and parameters are the compression of task data. Therefore, they could be strongly correlated and can be learned to predict one from the other. This paper proposes a novel neural network framework, SHIP (\textbf{Sh}uttle between the \textbf{I}nstructions and the \textbf{P}arameters), to model and learn the mutual mappings between the instructions and the parameters of LLMs. We verify that SHIP can effectively map one of the instructions/parameters to the other by evaluating it on the tasks of instruction deduction and induction. The results show that SHIP performs better than existing baseline methods in terms of deductive capabilities while significantly surpassing them in inductive capabilities. Moreover, SHIP can effectively combine the two mapping processes to perform excellent inductive reasoning. The code and data for this paper are released at https://anonymous.4open.science/r/Shuttle-Between-Instructions-Parameters/.

Figures (11)

• Figure 1: The concept of shuttling between the instructions and parameters, and the verification tasks.
• Figure 2: The framework of SHIP. The Training process is represented with filled colors and the inference process is represented with border colors.
• Figure 3: The loss curve of training SHIP.
• Figure 4: Analysis of SHIP's generalization ability and few-shot induction ability.
• Figure 5: The t-SNE result of latent $z$.