Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction

Rafael R. Baptista, André de Lima Salgado, Ricardo V. Godoy, Marcelo Becker, Thiago Boaventura, Gustavo J. G. Lahr

TL;DR

It is demonstrated that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.

Abstract

Leader-follower interaction is an important paradigm in human-robot interaction (HRI). Yet, assigning roles in real time remains challenging for resource-constrained mobile and assistive robots. While large language models (LLMs) have shown promise for natural communication, their size and latency limit on-device deployment. Small language models (SLMs) offer a potential alternative, but their effectiveness for role classification in HRI has not been systematically evaluated. In this paper, we present a benchmark of SLMs for leader-follower communication, introducing a novel dataset derived from a published database and augmented with synthetic samples to capture interaction-specific dynamics. We investigate two adaptation strategies: prompt engineering and fine-tuning, studied under zero-shot and one-shot interaction modes, compared with an untrained baseline. Experiments with Qwen2.5-0.5B reveal that zero-shot fine-tuning achieves robust classification performance (86.66% accuracy) while maintaining low latency (22.2 ms per sample), significantly outperforming baseline and prompt-engineered approaches. However, results also indicate a performance degradation in one-shot modes, where increased context length challenges the model's architectural capacity. These findings demonstrate that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.

Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction

TL;DR

It is demonstrated that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.

Abstract

Leader-follower interaction is an important paradigm in human-robot interaction (HRI). Yet, assigning roles in real time remains challenging for resource-constrained mobile and assistive robots. While large language models (LLMs) have shown promise for natural communication, their size and latency limit on-device deployment. Small language models (SLMs) offer a potential alternative, but their effectiveness for role classification in HRI has not been systematically evaluated. In this paper, we present a benchmark of SLMs for leader-follower communication, introducing a novel dataset derived from a published database and augmented with synthetic samples to capture interaction-specific dynamics. We investigate two adaptation strategies: prompt engineering and fine-tuning, studied under zero-shot and one-shot interaction modes, compared with an untrained baseline. Experiments with Qwen2.5-0.5B reveal that zero-shot fine-tuning achieves robust classification performance (86.66% accuracy) while maintaining low latency (22.2 ms per sample), significantly outperforming baseline and prompt-engineered approaches. However, results also indicate a performance degradation in one-shot modes, where increased context length challenges the model's architectural capacity. These findings demonstrate that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.
Paper Structure (10 sections, 4 figures, 1 table)

This paper contains 10 sections, 4 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. METHODS
  3. Dataset
  4. Language Model Evaluation
  5. Prompt Engineering
  6. Fine-tuning
  7. EXPERIMENTAL EVALUATION
  8. Classification Metrics
  9. Sentence Length Analysis
  10. CONCLUSION

Figures (4)

  • Figure 1: Edge-deployed leader–follower classification for assistive dyadic interaction. The system architecture selects the most appropriate low-level controller based on the social interation.
  • Figure 2: Overview of the dataset construction and model evaluation pipeline. The top section illustrates the creation of zero-shot and one-shot datasets, generated from DailyDialog samples and synthetic augmentation. The bottom section shows how these datasets were used to train and evaluate models under two adaptation strategies (prompt engineering and fine-tuning) for both zero-shot and one-shot interaction modes.
  • Figure 3: Accuracy dispersion across 30 runs for each interaction mode and adaptation strategy. Fine-tuned models consistently achieve higher median accuracy than baseline and prompt-engineered approaches, though the one-shot fine-tuned condition shows slightly greater variability.
  • Figure 4: Accuracy as a function of sentence length for the fine-tuned models for a) Zero-shot and b) One-shot. Error bars represent one standard deviation.