Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scenario-Based Field Testing of Drone Missions

Michael Vierhauser, Kristof Meixner, Stefan Biffl

TL;DR

Field testing of drone CPS faces environmental volatility and a simulator-to-reality gap, making structured, adaptable guidance essential. The authors propose Field Testing Scenario Management FiTS, which combines scenario-based requirements engineering and Behavior-Driven Development to define structured, reusable field test scenarios with role-specific guidance. FiTS comprises three integrated parts for design, execution, and analysis, and this paper mainly evaluates the design/managment component through three use cases and expert interviews. The feasibility study indicates FiTS is practical and beneficial for data collection, QA, and iterative improvement, with future work including automation, traceability to requirements, and broader empirical validation.

Abstract

Testing and validating Cyber-Physical Systems (CPSs) in the aerospace domain, such as field testing of drone rescue missions, poses challenges due to volatile mission environments, such as weather conditions. While testing processes and methodologies are well established, structured guidance and execution support for field tests are still weak. This paper identifies requirements for field testing of drone missions, and introduces the Field Testing Scenario Management (FiTS) approach for adaptive field testing guidance. FiTS aims to provide sufficient guidance for field testers as a foundation for efficient data collection to facilitate quality assurance and iterative improvement of field tests and CPSs. FiTS shall leverage concepts from scenario-based requirements engineering and Behavior-Driven Development to define structured and reusable test scenarios, with dedicated tasks and responsibilities for role-specific guidance. We evaluate FiTS by (i) applying it to three use cases for a search-and-rescue drone application to demonstrate feasibility and (ii) interviews with experienced drone developers to assess its usefulness and collect further requirements. The study results indicate FiTS to be feasible and useful to facilitate drone field testing and data analysis

Scenario-Based Field Testing of Drone Missions

TL;DR

Field testing of drone CPS faces environmental volatility and a simulator-to-reality gap, making structured, adaptable guidance essential. The authors propose Field Testing Scenario Management FiTS, which combines scenario-based requirements engineering and Behavior-Driven Development to define structured, reusable field test scenarios with role-specific guidance. FiTS comprises three integrated parts for design, execution, and analysis, and this paper mainly evaluates the design/managment component through three use cases and expert interviews. The feasibility study indicates FiTS is practical and beneficial for data collection, QA, and iterative improvement, with future work including automation, traceability to requirements, and broader empirical validation.

Abstract

Testing and validating Cyber-Physical Systems (CPSs) in the aerospace domain, such as field testing of drone rescue missions, poses challenges due to volatile mission environments, such as weather conditions. While testing processes and methodologies are well established, structured guidance and execution support for field tests are still weak. This paper identifies requirements for field testing of drone missions, and introduces the Field Testing Scenario Management (FiTS) approach for adaptive field testing guidance. FiTS aims to provide sufficient guidance for field testers as a foundation for efficient data collection to facilitate quality assurance and iterative improvement of field tests and CPSs. FiTS shall leverage concepts from scenario-based requirements engineering and Behavior-Driven Development to define structured and reusable test scenarios, with dedicated tasks and responsibilities for role-specific guidance. We evaluate FiTS by (i) applying it to three use cases for a search-and-rescue drone application to demonstrate feasibility and (ii) interviews with experienced drone developers to assess its usefulness and collect further requirements. The study results indicate FiTS to be feasible and useful to facilitate drone field testing and data analysis
Paper Structure (15 sections, 2 figures, 1 table)

This paper contains 15 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background & Example
  3. The FiTS Framework
  4. Test Management
  5. Test Execution
  6. Test Analysis
  7. Creation of Field Test Scenarios
  8. Validation - Scenario-based Feasibility Study
  9. Case Project: A Multi-sUAS Application
  10. Evaluation Setup
  11. Results
  12. Discussion
  13. Threats to Validity
  14. Related Work
  15. Conclusion and Future Work

Figures (2)

  • Figure 1: FiTS Framework overview.
  • Figure 2: Multi-drone field test equipment.