Occam's Razor and Bender and Koller's Octopus

TL;DR

This paper reframes the Bender-Koller debate on whether semantics can be learned from form alone by offering a lecture and activities that contrast the octopus analogy with counterarguments. It argues that scientific theory formation from observational data can proceed under plausible inductive biases and parsimony via Occam's razor, challenging the claim that LLMs cannot acquire world knowledge. The contribution is an instructor-ready teaching package that guides critical engagement with B&K and provides materials to stimulate discussion on data-driven meaning and learning of science. The work highlights the importance of examining inductive biases and structured pedagogy in NLP to assess claims about language understanding and generalization.

Abstract

We discuss the teaching of the discussion surrounding Bender and Koller's prominent ACL 2020 paper, "Climbing toward NLU: on meaning form, and understanding in the age of data" \cite{bender2020climbing}. We present what we understand to be the main contentions of the paper, and then recommend that the students engage with the natural counter-arguments to the claims in the paper. We attach teaching materials that we use to facilitate teaching this topic to undergraduate students.