Grounded learning for compositional vector semantics
Martha Lewis
TL;DR
The paper tackles the lack of cognitive grounding in tensor-based compositional distributional semantics by proposing a framework to implement grounded meanings within a biologically plausible spiking neural network (Nengo). It maps tensor-binding operations to binding via circular convolution and unbinding via circular correlation using semantic pointers, enabling role-filler-like composition in a neural substrate. A toy implementation (e.g., pet fish) demonstrates how adjectives, nouns, and verbs can be composed and queried within this framework, and explicit learning rules are provided to ground word representations from labelled images. This work offers a path toward environment-grounded, cognitively plausible language representations with potential applications in dialogue and interactive AI systems.
Abstract
Categorical compositional distributional semantics is an approach to modelling language that combines the success of vector-based models of meaning with the compositional power of formal semantics. However, this approach was developed without an eye to cognitive plausibility. Vector representations of concepts and concept binding are also of interest in cognitive science, and have been proposed as a way of representing concepts within a biologically plausible spiking neural network. This work proposes a way for compositional distributional semantics to be implemented within a spiking neural network architecture, with the potential to address problems in concept binding, and give a small implementation. We also describe a means of training word representations using labelled images.