Large Language Models are Limited in Out-of-Context Knowledge Reasoning

TL;DR

This work defines Out-of-Context Knowledge Reasoning (OCKR) and formalizes it for binary inference across Attributes ($A$) and Relations ($R$). It introduces the ID-OCKR benchmark with seven subsets, simple/hard levels, and a cross-lingual component to systematically probe OCKR in LLMs. Across multiple models and training regimes, the study finds that current LLMs struggle to perform OCKR outside of in-context prompts; reasoning training yields limited gains, while explicit retrieval helps only attribute knowledge and not relational knowledge, with cross-lingual transfer remaining weak. The findings highlight fundamental bottlenecks in knowledge retrieval and plan forward directions such as bidirectional training, semantic-aware retrieval, and insertion of planning aids to improve OCKR robustness and cross-lingual capabilities.

Abstract

Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. However, previous work challenges their out-of-context reasoning ability, i.e., the ability to infer information from their training data, instead of from the context or prompt. This paper focuses on a significant aspect of out-of-context reasoning: Out-of-Context Knowledge Reasoning (OCKR), which is to combine multiple knowledge to infer new knowledge. We designed a synthetic dataset with seven representative OCKR tasks to systematically assess the OCKR capabilities of LLMs. Using this dataset, we evaluated several LLMs and discovered that their proficiency in this aspect is limited, regardless of whether the knowledge is trained in a separate or adjacent training settings. Moreover, training the model to reason with reasoning examples does not result in significant improvement, while training the model to perform explicit knowledge retrieval helps for retrieving attribute knowledge but not the relation knowledge, indicating that the model's limited OCKR capabilities are due to difficulties in knowledge retrieval. Furthermore, we treat cross-lingual knowledge transfer as a distinct form of OCKR, and evaluate this ability. Our results show that the evaluated model also exhibits limited ability in transferring knowledge across languages.

Figures (2)

• Figure 1: In-Context vs Out-of-Context. In the In-Context scenario, the relevant data is provided in the prompt to allow the model to infer the answer. In the Out-of-Context scenario, the relevant data is included directly in the training data, and the model is then asked to infer the answer based on this training.
• Figure 2: The diagram shows the entities, attributes and relations in the dataset, for simple versions of the three reasoning patterns. Rectangles denote entities, ellipses indicate attributes, and edges represent relationships. Solid black lines represent knowledge in the training data, while dashed black lines represent knowledge in the test data. As the reasoning examples (Sec. \ref{['sec:reasoningTraining']}), a portion of the knowledge represented by the dashed black lines are provided to the training process for learning the corresponding inference patterns. The model is then tested on the knowledge represented by the remaining dashed black lines.