Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Learning Capabilities of Language Models using LEVERWORLDS

Eitan Wagner, Amir Feder, Omri Abend

TL;DR

LeverWorlds provides a controlled, physics-inspired framework to study how language models learn universal world structure versus instance-specific variability under sample constraints. The paper systematically compares classical estimators with Transformer-based approaches, revealing that structure-aware methods attain substantially better sample efficiency than LLMs without strong inductive biases. It also demonstrates that large LLMs in zero-shot or pure ICL settings struggle, but that a pipeline approach combining LLMs with classical models can yield promising improvements. Overall, LeverWorlds highlights a fundamental trade-off between flexibility and sample efficiency in world-model learning and points to hybrid strategies as a practical path forward.

Abstract

Learning a model of a stochastic setting often involves learning both general structure rules and specific properties of the instance. This paper investigates the interplay between learning the general and the specific in various learning methods, with emphasis on sample efficiency. We design a framework called {\sc LeverWorlds}, which allows the generation of simple physics-inspired worlds that follow a similar generative process with different distributions, and their instances can be expressed in natural language. These worlds allow for controlled experiments to assess the sample complexity of different learning methods. We experiment with classic learning algorithms as well as Transformer language models, both with fine-tuning and In-Context Learning (ICL). Our general finding is that (1) Transformers generally succeed in the task; but (2) they are considerably less sample efficient than classic methods that make stronger assumptions about the structure, such as Maximum Likelihood Estimation and Logistic Regression. This finding is in tension with the recent tendency to use Transformers as general-purpose estimators. We propose an approach that leverages the ICL capabilities of contemporary language models to apply simple algorithms for this type of data. Our experiments show that models currently struggle with the task but show promising potential.

Exploring the Learning Capabilities of Language Models using LEVERWORLDS

TL;DR

LeverWorlds provides a controlled, physics-inspired framework to study how language models learn universal world structure versus instance-specific variability under sample constraints. The paper systematically compares classical estimators with Transformer-based approaches, revealing that structure-aware methods attain substantially better sample efficiency than LLMs without strong inductive biases. It also demonstrates that large LLMs in zero-shot or pure ICL settings struggle, but that a pipeline approach combining LLMs with classical models can yield promising improvements. Overall, LeverWorlds highlights a fundamental trade-off between flexibility and sample efficiency in world-model learning and points to hybrid strategies as a practical path forward.

Abstract

Learning a model of a stochastic setting often involves learning both general structure rules and specific properties of the instance. This paper investigates the interplay between learning the general and the specific in various learning methods, with emphasis on sample efficiency. We design a framework called {\sc LeverWorlds}, which allows the generation of simple physics-inspired worlds that follow a similar generative process with different distributions, and their instances can be expressed in natural language. These worlds allow for controlled experiments to assess the sample complexity of different learning methods. We experiment with classic learning algorithms as well as Transformer language models, both with fine-tuning and In-Context Learning (ICL). Our general finding is that (1) Transformers generally succeed in the task; but (2) they are considerably less sample efficient than classic methods that make stronger assumptions about the structure, such as Maximum Likelihood Estimation and Logistic Regression. This finding is in tension with the recent tendency to use Transformers as general-purpose estimators. We propose an approach that leverages the ICL capabilities of contemporary language models to apply simple algorithms for this type of data. Our experiments show that models currently struggle with the task but show promising potential.
Paper Structure (32 sections, 6 equations, 6 figures, 1 table)

This paper contains 32 sections, 6 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. World Models
  4. In pretrained LLMs.
  5. Learning world models from examples.
  6. Integrating world models and language.
  7. Finetuning and In-Context Learning
  8. Bias-variance Tradeoff
  9. The LeverWorlds Environment
  10. Setting
  11. Evaluation
  12. Distribution Similarity
  13. Structure Similarity
  14. Experiments
  15. Baseline Methods
  16. ...and 17 more sections

Figures (6)

  • Figure 1: Overview of our experiments. First, we generate a physical model, then we sample from the model and train a language model to predict the output. We then evaluate the model's probability estimations.
  • Figure 2: Causal graph for balance on a lever. Different worlds differ by the number of objects, by the optional use of density and volume, and by whether the intermediate variables are observed or not.
  • Figure 3: Results for OPT models. In the first row are the results for world-1 and in the second are the results for world-3. In cases, we plot the metric as a function of the number of training samples.
  • Figure 4: Results for Logistic Regression models.
  • Figure 5: Results for MLE models.
  • ...and 1 more figures