Mutation Testing for Industrial Robotic Systems

TL;DR

The paper tackles the challenge that traditional mutation testing fails to capture the semantics of industrial robotic systems (IRS). It introduces domain-specific mutation operators that act on high-level write/read actions for movements, gripper control, and sensor readings, and evaluates them in a Gazebo/ROS pick-and-place scenario. The results show that domain-specific mutants are more informative and avoid many invalid or equivalent mutants, achieving mutation scores in the range of $77\%$ to $81\%$, thereby improving test-suite adequacy for IRS. This work suggests that mutation testing can meaningfully enhance safety and reliability in industrial robotics by focusing on realistic faults and scalable evaluation.

Abstract

Industrial robotic systems (IRS) are increasingly deployed in diverse environments, where failures can result in severe accidents and costly downtime. Ensuring the reliability of the software controlling these systems is therefore critical. Mutation testing, a technique widely used in software engineering, evaluates the effectiveness of test suites by introducing small faults, or mutants, into the code. However, traditional mutation operators are poorly suited to robotic programs, which involve message-based commands and interactions with the physical world. This paper explores the adaptation of mutation testing to IRS by defining domain-specific mutation operators that capture the semantics of robot actions and sensor readings. We propose a methodology for generating meaningful mutants at the level of high-level read and write operations, including movement, gripper actions, and sensor noise injection. An empirical study on a pick-and-place scenario demonstrates that our approach produces more informative mutants and reduces the number of invalid or equivalent cases compared to conventional operators. Results highlight the potential of mutation testing to enhance test suite quality and contribute to safer, more reliable industrial robotic systems.

Figures (7)

• Figure 1: The main components of an industrial robotic system.
• Figure 2: Examples of programs written in two IRPLs: ABB's RAPID (left) and KUKA's KRL (right) POGLIANI2020.
• Figure 3: A screenshot of ROS and Gazebo simulating the operation of a Kinova Gen3 gripper arm.
• Figure 4: A simple program and a possible mutant.
• Figure 5: The pick-and-place scenario used in our experiment, with the localization of points where mutation operators are applied.