Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networks

TL;DR

The paper critically reevaluates Lake & Baroni's meta-learning approach to systematic compositionality, arguing that current neural meta-learning yields human-like, structure-sensitive generalization only under narrow conditions and often via memorization rather than genuine rule discovery. It reaffirms Fodor & Pylyshyn's legacy, showing that without explicit symbolic representations and iterative rule validation, neural networks fail to achieve true systematic compositionality. The authors advocate for evaluation frameworks and training regimes that emphasize reflective, iterative reasoning and symbolics, potentially via hybrid neuro-symbolic architectures. The work highlights the need to bridge connectionist and symbolic approaches to realize robust, scalable, and explainable systematic generalization in AI systems with real-world task relevance.

Abstract

Strong meta-learning capabilities for systematic compositionality are emerging as an important skill for navigating the complex and changing tasks of today's world. However, in presenting models for robust adaptation to novel environments, it is important to refrain from making unsupported claims about the performance of meta-learning systems that ultimately do not stand up to scrutiny. While Fodor and Pylyshyn famously posited that neural networks inherently lack this capacity as they are unable to model compositional representations or structure-sensitive operations, and thus are not a viable model of the human mind, Lake and Baroni recently presented meta-learning as a pathway to compositionality. In this position paper, we critically revisit this claim and highlight limitations in the proposed meta-learning framework for compositionality. Our analysis shows that modern neural meta-learning systems can only perform such tasks, if at all, under a very narrow and restricted definition of a meta-learning setup. We therefore claim that `Fodor and Pylyshyn's legacy' persists, and to date, there is no human-like systematic compositionality learned in neural networks.

Figures (1)

• Figure 1: The challenge of claiming and testing systematic compositionality. Given the undisputed importance of compositional representations and structure-sensitive operations for systematic compositionality, their evaluation remains crucial and challenging. While structure-sensitivity can be assessed by comprehensive OOD testing, the investigation of representations requires some inspectable model architecture.

Theorems & Definitions (1)

• Claim 1