Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions

TL;DR

This work addresses the challenge that instruction-tuned language models struggle to generalize to unseen tasks. It introduces Instructive Decoding (ID), a decoding-time method that uses noisy instruction variants to anchor model predictions and contrasts logits from the original versus noisy instructions, all without updating parameters. Across multiple Tk-Instruct and OpenSNI models on SupNatInst and UnNatInst, ID yields consistent improvements in Rouge-L and related metrics, with the Opposite-noise variant often delivering the strongest gains. Analyses show an anchoring effect where degraded noisy instructions can lead to larger improvements when used in ID, and the method also improves instruction adherence and semantic coherence. These findings suggest a practical, scalable approach to bolster instruction following in LMs without retraining, with implications for efficiency, robustness, and applicability across diverse tasks.

Abstract

While instruction-tuned language models have demonstrated impressive zero-shot generalization, these models often struggle to generate accurate responses when faced with instructions that fall outside their training set. This paper presents Instructive Decoding (ID), a simple yet effective approach that augments the efficacy of instruction-tuned models. Specifically, ID adjusts the logits for next-token prediction in a contrastive manner, utilizing predictions generated from a manipulated version of the original instruction, referred to as a noisy instruction. This noisy instruction aims to elicit responses that could diverge from the intended instruction yet remain plausible. We conduct experiments across a spectrum of such noisy instructions, ranging from those that insert semantic noise via random words to others like 'opposite' that elicit the deviated responses. Our approach achieves considerable performance gains across various instruction-tuned models and tasks without necessitating any additional parameter updates. Notably, utilizing 'opposite' as the noisy instruction in ID, which exhibits the maximum divergence from the original instruction, consistently produces the most significant performance gains across multiple models and tasks.

Figures (15)

• Figure 1: Overview of Instructive Decoding (ID). The example in this figure is from task442_com_qa_paraphrase_question_generation in SupNatInstsni_dataset. The original response not only fails to meet the task requirements (Question Rewriting) but also contains incorrect information. In contrast, ID generates a more relevant response by refining its next-token predictions based on the noisy instruction (here, opposite prompting is used for ID).
• Figure 2: Zero-shot Rouge-L comparison on the SupNatInst heldout dataset sni_dataset. Models not instruction-tuned on SupNatInst are in blue dotted boxes, while those instruction-tuned are in green.
• Figure 2: Rouge-L scores cross-evaluated across different models and datasets.
• Figure 3: Full-text examples for a collection of noisy instructions for instructive decoding on task442_com_qa_paraphrase_question_generation.
• Figure 4: (a) Correlation between performance degradation with noisy instructions and improvement with those used in ID; (b) comparative winning rates of Base vs. Ours. on held-out tasks. The blue bars show base method wins, while the green bars indicate our method's dominance.