Comparing Effectiveness and Efficiency of Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) Tools in a Large Java-based System

TL;DR

This study evaluates Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) in a large Java-based OpenMRS system, comparing them to established baselines SAST, DAST, SMPT, and EMPT following Elder et al. The authors find that IAST achieves strong overall effectiveness and competitive efficiency (average Vulnerabilities per Hour $VpH$ of $2.14$, with IAST-2 reaching $2.98$) and identifies many unique vulnerabilities, including eight OWASP Top Ten categories, while EMPT and SAST still yield stronger performance on certain high-severity vulnerabilities. RASP demonstrates meaningful runtime protection by preventing exploitation—averaging about $44$ prevented vulnerabilities per run—but does not replace external vulnerability detection in testing. Overall, IAST complements existing techniques and, together with RASP, supports a defense-in-depth approach in enterprise-scale web applications, offering practitioners guidance on tool selection based on vulnerability type priorities and resource constraints. The work highlights practical metrics such as vulnerability per hour and OWASP/CWE mappings to quantify effectiveness, enabling more evidence-based security planning in CI/CD contexts. $VpH$ and CVE/OWASP mappings are central to the analysis, emphasizing the need for multi-tool strategies in complex software ecosystems.

Abstract

Security resources are scarce, and practitioners need guidance in the effective and efficient usage of techniques and tools available in the cybersecurity industry. Two emerging tool types, Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP), have not been thoroughly evaluated against well-established counterparts such as Dynamic Application Security Testing (DAST) and Static Application Security Testing (SAST). The goal of this research is to aid practitioners in making informed choices about the use of Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) tools through an analysis of their effectiveness and efficiency in comparison with different vulnerability detection and prevention techniques and tools. We apply IAST and RASP on OpenMRS, an open-source Java-based online application. We compare the efficiency and effectiveness of IAST and RASP with techniques applied on OpenMRS in prior work. We measure efficiency and effectiveness in terms of the number and type of vulnerabilities detected and prevented per hour. Our study shows IAST performed relatively well compared to other techniques, performing second-best in both efficiency and effectiveness. IAST detected eight Top-10 OWASP security risks compared to nine by SMPT and seven for EMPT, DAST, and SAST. IAST found more vulnerabilities than SMPT. The efficiency of IAST (2.14 VpH) is second to only EMPT (2.22 VpH). These findings imply that our study benefited from using IAST when conducting black-box security testing. In the context of a large, enterprise-scale web application such as OpenMRS, RASP does not replace vulnerability detection, while IAST is a powerful tool that complements other techniques.

Figures (3)

• Figure 1: Abstraction of IAST and RASP (Java Example)
• Figure 2: A high-level overview of our approach.
• Figure 3: Severity of IAST Vulnerabilities grouped into 2021 OWASP Top Ten Categories