ROBUST: 221 Bugs in the Robot Operating System
Christopher S. Timperley, Gijs van der Hoorn, André Santos, Harshavardhan Deshpande, Andrzej Wąsowski
TL;DR
ROBUST presents a dataset of 221 bugs in Robot Operating System software, collected from seven ROS subject systems and reproduced in historically accurate Docker environments. Using artifact-driven methods and grounded theory, the authors classify faults and failures, revealing that most fixes are small and localized despite diverse and complex root causes. The work introduces a time-machine style infrastructure (rosinstall/ROS distributions, BugZoo) to faithfully reproduce historical contexts, enabling reproducible QA research and benchmark development for robotics software. These findings underscore the need for enhanced CI/test practices, better handling of configuration and evolution, and tooling that supports cross-language and domain-specific analysis in robotics.
Abstract
As robotic systems such as autonomous cars and delivery drones assume greater roles and responsibilities within society, the likelihood and impact of catastrophic software failure within those systems is increased.To aid researchers in the development of new methods to measure and assure the safety and quality of robotics software, we systematically curated a dataset of 221 bugs across 7 popular and diverse software systems implemented via the Robot Operating System (ROS). We produce historically accurate recreations of each of the 221 defective software versions in the form of Docker images, and use a grounded theory approach to examine and categorize their corresponding faults, failures, and fixes. Finally, we reflect on the implications of our findings and outline future research directions for the community.