Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AIM: Automated Input Set Minimization for Metamorphic Security Testing

Nazanin Bayati Chaleshtari, Yoann Marquer, Fabrizio Pastore, Lionel C. Briand

TL;DR

This work proposes AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities, and evaluated the effectiveness of AIM on two well-known Web systems, Jenkins and Joomla, with documented vulnerabilities.

Abstract

Although the security testing of Web systems can be automated by generating crafted inputs, solutions to automate the test oracle, i.e., vulnerability detection, remain difficult to apply in practice. Specifically, though previous work has demonstrated the potential of metamorphic testing, security failures can be determined by metamorphic relations that turn valid inputs into malicious inputs, metamorphic relations are typically executed on a large set of inputs, which is time-consuming and thus makes metamorphic testing impractical. We propose AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities. AIM includes a clustering-based black-box approach, to identify similar inputs based on their security properties. It also relies on a novel genetic algorithm to efficiently select diverse inputs while minimizing their total cost. Further, it contains a problem-reduction component to reduce the search space and speed up the minimization process. We evaluated the effectiveness of AIM on two well-known Web systems, Jenkins and Joomla, with documented vulnerabilities. We compared AIM's results with four baselines involving standard search approaches. Overall, AIM reduced metamorphic testing time by 84% for Jenkins and 82% for Joomla, while preserving the same level of vulnerability detection. Furthermore, AIM significantly outperformed all the considered baselines regarding vulnerability coverage.

AIM: Automated Input Set Minimization for Metamorphic Security Testing

TL;DR

This work proposes AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities, and evaluated the effectiveness of AIM on two well-known Web systems, Jenkins and Joomla, with documented vulnerabilities.

Abstract

Although the security testing of Web systems can be automated by generating crafted inputs, solutions to automate the test oracle, i.e., vulnerability detection, remain difficult to apply in practice. Specifically, though previous work has demonstrated the potential of metamorphic testing, security failures can be determined by metamorphic relations that turn valid inputs into malicious inputs, metamorphic relations are typically executed on a large set of inputs, which is time-consuming and thus makes metamorphic testing impractical. We propose AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities. AIM includes a clustering-based black-box approach, to identify similar inputs based on their security properties. It also relies on a novel genetic algorithm to efficiently select diverse inputs while minimizing their total cost. Further, it contains a problem-reduction component to reduce the search space and speed up the minimization process. We evaluated the effectiveness of AIM on two well-known Web systems, Jenkins and Joomla, with documented vulnerabilities. We compared AIM's results with four baselines involving standard search approaches. Overall, AIM reduced metamorphic testing time by 84% for Jenkins and 82% for Joomla, while preserving the same level of vulnerability detection. Furthermore, AIM significantly outperformed all the considered baselines regarding vulnerability coverage.
Paper Structure (68 sections, 25 equations, 8 figures, 15 tables, 3 algorithms)

This paper contains 68 sections, 25 equations, 8 figures, 15 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Metamorphic Testing
  4. Metamorphic Security Testing
  5. Test Suite Minimization
  6. Clustering
  7. K-means
  8. DBSCAN
  9. HDBSCAN
  10. Many Objective Optimization
  11. Pareto Front
  12. Solving Many-Objective Problems
  13. Problem Definition
  14. Assumptions and Goals
  15. A Many-Objective Problem
  16. ...and 53 more sections

Figures (8)

  • Figure 1: MR for CWE 668: Exposure of resource to wrong sphere cwe668
  • Figure 2: Linear regression between the number of executed actions and the execution time of a metamorphic relation. Each input is represented by a green dot, while the blue line depicts the linear regression model.
  • Figure 3: Activity diagram of the Automated Input Minimizer(AIM) approach.
  • Figure 4: The URL distance between http://hostname/login and http://hostname/job/try1/lastBuild is $1 + 3 = 4$.
  • Figure 5: Inputs covering one objective in common (left) are connected in the corresponding overlapping graph (right).
  • ...and 3 more figures