Instruction Boundary: Quantifying Biases in LLM Reasoning under Various Coverage

TL;DR

This paper formalizes Instruction Boundary, a phenomenon where LLM reasoning is biased by the type and completeness of prompts, especially in the presence of sparse labels. It introduces BiasDetector to quantify how well models identify sparse labels under varying instruction coverage and evaluates five mainstream LLMs across eight prompt-settings using nine datasets. The findings reveal persistent reasoning biases even with complete instructions and highlight that redundant and insufficient prompts can amplify these biases, affecting reliability more than raw accuracy. The work discusses mitigation strategies and emphasizes the need for coverage-aware calibration to improve LLM trustworthiness in real-world applications.

Abstract

Nowadays, automatically generated datasets are increasingly used in LLM reasoning tasks; however, large-scale corpora often contain inherent flaws. For example, a single-choice question may include none or multiple correct options, while true-or-false questions may involve vague or unverifiable statements. We refer to these exceptional answer forms as sparse labels. To compare LLMs' ability to recognize various question forms and produce correct answers, we investigate how different instruction formats can either facilitate or mislead LLM reasoning ability. We introduce the concept of Instruction Boundary, which systematically analyzes how different levels of prompt coverage -- sufficient, redundant, or insufficient -- can lead to reasoning biases and performance changes in LLMs. To examine this phenomenon, we design eight experimental settings across five dataset forms. We further propose BiasDetector, a unified framework that quantifies LLMs' ability to identify sparse labels under different kinds of Instruction Boundary conditions. Evaluations on five mainstream LLMs show that, despite their seemingly high accuracy, substantial reasoning biases persist in many downstream tasks as a direct consequence of prompt coverage. We analyze the impact of these biases and outline possible mitigation strategies. Our findings highlight not only the importance of addressing sparse labels, but also the need for developers to recognize and mitigate the risks introduced by Instruction Boundary.

Figures (7)

• Figure 1: Brief illustrations of potential threat of LLM reasoning biases (Above three). As shown in the picture, even though LLMs can sometimes achieve a 50% or more accuracy in name, however, their understanding towards problems remains limited, which undermines the trustworthiness. And comparisons of previous work Knowledge Boundary and ours definition of Instruction Boundary (Below). While previous work mainly focuses on the robustness of LLMs in downstream tasks, we incorporate reasoning biases together with instruction coverage to better reflect real-world scenarios.
• Figure 2: General pipeline of our proposed framework BiasDetector. First of all, we decompose the phenomenon by settings and define problem types regarding different aspect of Instruction Boundary. Then we apply five LLMs from different brands to generate biased reasoning under these settings. Finally by testing their robustness and accuracy regarding different aspects, we discover problems and thus offer suggestions for future LLM improvements.
• Figure 3: Results of Accuracy of six LLMs experiments under Disturbing Miscellany setting, the accuracy is dropping as the number of options is increasing according to the x-axis number of options (left) and general Accuracy among three datasets forms (middle), and Label Output Distribution (OR) under Vanilla Scenario settings (right). Standardized with the scale of the rightmost figure as the reference.
• Figure 4: Comparisons of Conformity, Few-shot and Disturbing Miscellany settings. The Generalization Rate (GR), accuracy difference between redundant options and vanilla setting is depicted. For the two in the middle are under conformity setting Output Rate (OR) and difference of two kinds among five LLMs experiments under conformity evaluations. Metrics details are elaborated in Section \ref{['Metrics']}. The eightieth chart is $Acc_{b}$, GR and SR difference of Few-shot setting between tense and sparse label setting, from which we can tell Gemini-2.0-flash is largely influenced by Few-shot example types.
• Figure 5: General leader board of related topic open-source benchmarks(left), and inner settings comparisons among five LLMs used in our benchmark(right). The metric is Robustness Score (RS) as explained previously in Section \ref{['Metrics']}. For benchmark-wise comparisons, we show the ranking under Vanilla Scenario. Detailed values can be found in Table \ref{['tab:detailed-rs']}.