Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Test Data Generation for MC/DC with OCL and Search

Hassan Sartaj, Muhammad Zohaib Iqbal, Atif Aftab Ahmed Jilani, Muhammad Uzair Khan

TL;DR

The paper tackles automated MC/DC-compliant test data generation for OCL constraints in model-based testing of avionics software under DO-178C. It introduces a dual strategy that combines Case-based Reasoning (CBR) and range reduction to reformulate MC/DC constraints and seed searches for efficient solutions, using AVM as the underlying solver. Empirical evaluation across six diverse case studies (129 constraints) shows that the combined approach AVMrc delivers the highest success rates and broad MC/DC coverage, outperforming single-method variants and baseline search, while trading off some runtime against constraint solvers that are faster but solve fewer constraints. The work also benchmarks against UMLtoCSP and PLEDGE, showing superior constraint-solving coverage for AVM-based methods, and provides extensive discussion of threats to validity and applicability to industrial-scale MBT workflows.

Abstract

System-level testing of avionics software systems requires compliance with different international safety standards such as DO-178C. An important consideration of the avionics industry is automated test data generation according to the criteria suggested by safety standards. One of the recommended criteria by DO-178C is the modified condition/decision coverage (MC/DC) criterion. The current model-based test data generation approaches use constraints written in Object Constraint Language (OCL), and apply search techniques to generate test data. These approaches either do not support MC/DC criterion or suffer from performance issues while generating test data for large-scale avionics systems. In this paper, we propose an effective way to automate MC/DC test data generation during model-based testing. We develop a strategy that utilizes case-based reasoning (CBR) and range reduction heuristics designed to solve MC/DC-tailored OCL constraints. We performed an empirical study to compare our proposed strategy for MC/DC test data generation using CBR, range reduction, both CBR and range reduction, with an original search algorithm, and random search. We also empirically compared our strategy with existing constraint-solving approaches. The results show that both CBR and range reduction for MC/DC test data generation outperform the baseline approach. Moreover, the combination of both CBR and range reduction for MC/DC test data generation is an effective approach compared to existing constraint solvers.

Efficient Test Data Generation for MC/DC with OCL and Search

TL;DR

The paper tackles automated MC/DC-compliant test data generation for OCL constraints in model-based testing of avionics software under DO-178C. It introduces a dual strategy that combines Case-based Reasoning (CBR) and range reduction to reformulate MC/DC constraints and seed searches for efficient solutions, using AVM as the underlying solver. Empirical evaluation across six diverse case studies (129 constraints) shows that the combined approach AVMrc delivers the highest success rates and broad MC/DC coverage, outperforming single-method variants and baseline search, while trading off some runtime against constraint solvers that are faster but solve fewer constraints. The work also benchmarks against UMLtoCSP and PLEDGE, showing superior constraint-solving coverage for AVM-based methods, and provides extensive discussion of threats to validity and applicability to industrial-scale MBT workflows.

Abstract

System-level testing of avionics software systems requires compliance with different international safety standards such as DO-178C. An important consideration of the avionics industry is automated test data generation according to the criteria suggested by safety standards. One of the recommended criteria by DO-178C is the modified condition/decision coverage (MC/DC) criterion. The current model-based test data generation approaches use constraints written in Object Constraint Language (OCL), and apply search techniques to generate test data. These approaches either do not support MC/DC criterion or suffer from performance issues while generating test data for large-scale avionics systems. In this paper, we propose an effective way to automate MC/DC test data generation during model-based testing. We develop a strategy that utilizes case-based reasoning (CBR) and range reduction heuristics designed to solve MC/DC-tailored OCL constraints. We performed an empirical study to compare our proposed strategy for MC/DC test data generation using CBR, range reduction, both CBR and range reduction, with an original search algorithm, and random search. We also empirically compared our strategy with existing constraint-solving approaches. The results show that both CBR and range reduction for MC/DC test data generation outperform the baseline approach. Moreover, the combination of both CBR and range reduction for MC/DC test data generation is an effective approach compared to existing constraint solvers.
Paper Structure (35 sections, 7 equations, 4 figures, 6 tables)

This paper contains 35 sections, 7 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Model-based Testing
  4. Modified Condition/Decision Coverage (MC/DC)
  5. Search-based Software Engineering (SBSE)
  6. OCL Constraint Solving
  7. Case-based Reasoning
  8. Running Example
  9. Strategy to Generate Test Data for MC/DC
  10. OCL Constraints Reformulation for MC/DC
  11. Handling Negation for OCL
  12. Logical Operations Reformulation
  13. Identifying Conflicting MC/DC Constraints
  14. Solving MC/DC Constraints using Case-based Reasoning
  15. Solving MC/DC Constraints using Range Reduction
  16. ...and 20 more sections

Figures (4)

  • Figure 1: An excerpt of GCS class diagram
  • Figure 2: An overall view of the proposed MC/DC strategy
  • Figure 3: Boxplots showing the median success rates achieved by AVMo, AVMc, AVMr, AVMrc, and RS for each case study.
  • Figure 4: Line graphs showing the time taken by AVMo, AVMc, AVMr, AVMrc, and RS to solve MC/DC constraints for each case study across 100 evaluations.