Towards Interactive Language Modeling

TL;DR

The paper argues that interactive, teacher–student dynamics can enhance language modeling by introducing explicit interaction and a data-budgeted teaching paradigm inspired by caregiver–child communication. It establishes a concrete road map for interactive language modeling, then implements and evaluates a proof-of-concept where a pretrained teacher LM selects a small set of sentences to train a student LM under a fixed transmission budget, using REINFORCE and GumbelTopK for data selection. Experiments on artificial language tasks show the teacher can learn near-oracle didactic strategies, with performance gains reflected in held-out exam perplexity and the importance of embedding choice for signaling structure/domain differences. The work highlights interpretability, data-efficiency, and potential downstream benefits while outlining computational and methodological directions for a broader research agenda in interactive LM.

Abstract

Interaction between caregivers and children plays a critical role in human language acquisition and development. Given this observation, it is remarkable that explicit interaction plays little to no role in artificial language modeling -- which also targets the acquisition of human language, yet by artificial models. Moreover, an interactive approach to language modeling has the potential to make language models substantially more versatile and to considerably impact downstream applications. Motivated by these considerations, we pioneer the space of interactive language modeling. As a first contribution we present a road map in which we detail the steps that need to be taken towards interactive language modeling. We then lead by example and take the first steps on this road map, showing the initial feasibility of our approach. As such, this work aims to be the start of a larger research agenda on interactive language modeling.

Figures (8)

• Figure 1: Teacher-Student setup for interactive language modeling.
• Figure 2: Teacher-student loop as used in this work.
• Figure 3: Results Task 1 -- Different domains. Plots for different numbers of students per teacher. Results per setting reported as average and standard deviation over five random seeds. x-axis of lower plots bound to $40$ as the teacher had already converged by then.
• Figure 4: T-SNE plots for different sentence representations for different tasks.
• Figure 5: Results Task 2 -- Plots for 12 students per teacher. Results per setting reported as average and standard deviation over five random seeds.