Implicit In-Context Learning: Evidence from Artificial Language Experiments
Xiaomeng Ma, Qihui Xu
TL;DR
The study investigates whether large language models exhibit human-like implicit learning during inference by adapting three classical artificial language experiments across morphology, morphosyntax, and syntax to gpt-4o and o3-mini. It demonstrates domain-specific alignment: o3-mini shows probabilistic morphology learning similar to humans, while gpt-4o displays type-frequency sensitivity; both models align with humans in certain syntactic tasks but diverge in others. Across experiments, the work highlights that cognitive-science paradigms can rigorously probe implicit learning in LLMs and reveals architecture-dependent trade-offs between implicit learning and explicit reasoning. The findings provide a framework for benchmarking implicit-learning capabilities in LLMs and guide future model design and hypothesis-driven research in cognitive and computational linguistics.
Abstract
Humans acquire language through implicit learning, absorbing complex patterns without explicit awareness. While LLMs demonstrate impressive linguistic capabilities, it remains unclear whether they exhibit human-like pattern recognition during in-context learning at inferencing level. We adapted three classic artificial language learning experiments spanning morphology, morphosyntax, and syntax to systematically evaluate implicit learning at inferencing level in two state-of-the-art OpenAI models: gpt-4o and o3-mini. Our results reveal linguistic domain-specific alignment between models and human behaviors, o3-mini aligns better in morphology while both models align in syntax.