Small but Significant: On the Promise of Small Language Models for Accessible AIED

TL;DR

The paper argues that the education AI community's focus on GPT-scale LLMs risks excluding resource-constrained institutions from high-quality AI tools. It positions Small Language Models (SLMs), exemplified by Phi-2 (2.7B parameters) trained on textbook-quality data, as efficient, private, and deployable on consumer hardware, enabling equitable access. Through a KC discovery case study, the authors demonstrate that Phi-2 can outperform instructional experts and GPT-4o when its potential is exploited via a PMI-like similarity measure and clustering. The work advocates a balanced approach that integrates SLMs with LLMs to advance accessible, ethical, and effective AIED tools for diverse educational contexts.

Abstract

GPT has become nearly synonymous with large language models (LLMs), an increasingly popular term in AIED proceedings. A simple keyword-based search reveals that 61% of the 76 long and short papers presented at AIED 2024 describe novel solutions using LLMs to address some of the long-standing challenges in education, and 43% specifically mention GPT. Although LLMs pioneered by GPT create exciting opportunities to strengthen the impact of AI on education, we argue that the field's predominant focus on GPT and other resource-intensive LLMs (with more than 10B parameters) risks neglecting the potential impact that small language models (SLMs) can make in providing resource-constrained institutions with equitable and affordable access to high-quality AI tools. Supported by positive results on knowledge component (KC) discovery, a critical challenge in AIED, we demonstrate that SLMs such as Phi-2 can produce an effective solution without elaborate prompting strategies. Hence, we call for more attention to developing SLM-based AIED approaches.