Identifying Uncertainty in Self-Adaptive Robotics with Large Language Models

TL;DR

The paper addresses the challenge of identifying uncertainties in self-adaptive robotics across the software engineering lifecycle. It proposes a systematic approach using role-based prompting of 10 advanced LLMs to extract uncertainty-related insights from four industrial use cases, evaluated with practitioner feedback. Results show that practitioners agree with 63–88% of LLM-generated uncertainty responses and report domain understanding and practical usefulness, including the ability to surface overlooked uncertainties. The work provides actionable recommendations, such as cross-checking with a second LLM and cross-phase collaboration, and outlines future directions toward a holistic uncertainty identification framework and taxonomy for self-adaptive robotics.

Abstract

Future self-adaptive robots are expected to operate in highly dynamic environments while effectively managing uncertainties. However, identifying the sources and impacts of uncertainties in such robotic systems and defining appropriate mitigation strategies is challenging due to the inherent complexity of self-adaptive robots and the lack of comprehensive knowledge about the various factors influencing uncertainty. Hence, practitioners often rely on intuition and past experiences from similar systems to address uncertainties. In this article, we evaluate the potential of large language models (LLMs) in enabling a systematic and automated approach to identify uncertainties in self-adaptive robotics throughout the software engineering lifecycle. For this evaluation, we analyzed 10 advanced LLMs with varying capabilities across four industrial-sized robotics case studies, gathering the practitioners' perspectives on the LLM-generated responses related to uncertainties. Results showed that practitioners agreed with 63-88% of the LLM responses and expressed strong interest in the practicality of LLMs for this purpose.

Figures (2)

• Figure 1: Industrial context (left) and evaluation methodology overview (right). In the workflow, rounded rectangles indicate our activities, ovals represent individual activities by the practitioners, and hexagons denote collaborative activities.
• Figure 2: Results for all LLMs across four case studies, presenting the practitioners’ assessments of LLMs’ effectiveness in identifying uncertainties.