Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ROSA: Finding Backdoors with Fuzzing

Dimitri Kokkonis, Michaël Marcozzi, Emilien Decoux, Stefano Zacchiroli

TL;DR

ROSA tackles the problem of detecting code-level backdoors by combining a state-of-the-art graybox fuzzer (AFL++) with a novel metamorphic test oracle that signals runtime backdoor triggers. It introduces Rosarum, the first openly available benchmark containing 17 backdoors (7 authentic and 10 synthetic) to evaluate detection tools across diverse programs. Empirical evaluation shows ROSA achieves robustness and speed comparable to classical fuzzing, detecting all backdoors in about 1 hour and 30 minutes on average, with a semi-automated vetting step that minimizes manual effort. Compared with the sole competing tool Stringer, Rosa covers a wider range of backdoors with far fewer false positives, albeit with slower overall detection, offering a practical path toward scalable automated code auditing for software dependencies and firmware.

Abstract

A code-level backdoor is a hidden access, programmed and concealed within the code of a program. For instance, hard-coded credentials planted in the code of a file server application would enable maliciously logging into all deployed instances of this application. Confirmed software supply chain attacks have led to the injection of backdoors into popular open-source projects, and backdoors have been discovered in various router firmware. Manual code auditing for backdoors is challenging and existing semi-automated approaches can handle only a limited scope of programs and backdoors, while requiring manual reverse-engineering of the audited (binary) program. Graybox fuzzing (automated semi-randomized testing) has grown in popularity due to its success in discovering vulnerabilities and hence stands as a strong candidate for improved backdoor detection. However, current fuzzing knowledge does not offer any means to detect the triggering of a backdoor at runtime. In this work we introduce ROSA, a novel approach (and tool) which combines a state-of-the-art fuzzer (AFL++) with a new metamorphic test oracle, capable of detecting runtime backdoor triggers. To facilitate the evaluation of ROSA, we have created ROSARUM, the first openly available benchmark for assessing the detection of various backdoors in diverse programs. Experimental evaluation shows that ROSA has a level of robustness, speed and automation similar to classical fuzzing. It finds all 17 authentic or synthetic backdooors from ROSARUM in 1h30 on average. Compared to existing detection tools, it can handle a diversity of backdoors and programs and it does not rely on manual reverse-engineering of the fuzzed binary code.

ROSA: Finding Backdoors with Fuzzing

TL;DR

ROSA tackles the problem of detecting code-level backdoors by combining a state-of-the-art graybox fuzzer (AFL++) with a novel metamorphic test oracle that signals runtime backdoor triggers. It introduces Rosarum, the first openly available benchmark containing 17 backdoors (7 authentic and 10 synthetic) to evaluate detection tools across diverse programs. Empirical evaluation shows ROSA achieves robustness and speed comparable to classical fuzzing, detecting all backdoors in about 1 hour and 30 minutes on average, with a semi-automated vetting step that minimizes manual effort. Compared with the sole competing tool Stringer, Rosa covers a wider range of backdoors with far fewer false positives, albeit with slower overall detection, offering a practical path toward scalable automated code auditing for software dependencies and firmware.

Abstract

A code-level backdoor is a hidden access, programmed and concealed within the code of a program. For instance, hard-coded credentials planted in the code of a file server application would enable maliciously logging into all deployed instances of this application. Confirmed software supply chain attacks have led to the injection of backdoors into popular open-source projects, and backdoors have been discovered in various router firmware. Manual code auditing for backdoors is challenging and existing semi-automated approaches can handle only a limited scope of programs and backdoors, while requiring manual reverse-engineering of the audited (binary) program. Graybox fuzzing (automated semi-randomized testing) has grown in popularity due to its success in discovering vulnerabilities and hence stands as a strong candidate for improved backdoor detection. However, current fuzzing knowledge does not offer any means to detect the triggering of a backdoor at runtime. In this work we introduce ROSA, a novel approach (and tool) which combines a state-of-the-art fuzzer (AFL++) with a new metamorphic test oracle, capable of detecting runtime backdoor triggers. To facilitate the evaluation of ROSA, we have created ROSARUM, the first openly available benchmark for assessing the detection of various backdoors in diverse programs. Experimental evaluation shows that ROSA has a level of robustness, speed and automation similar to classical fuzzing. It finds all 17 authentic or synthetic backdooors from ROSARUM in 1h30 on average. Compared to existing detection tools, it can handle a diversity of backdoors and programs and it does not rely on manual reverse-engineering of the fuzzed binary code.
Paper Structure (38 sections, 2 figures, 2 tables)

This paper contains 38 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Code-level backdoors
  4. Definition and scope
  5. Occurrence in the real world
  6. Detection
  7. Graybox and metamorphic fuzzing
  8. Graybox fuzzing
  9. The AFL++ graybox fuzzer
  10. Binary fuzzing with AFL++
  11. Guessing magic bytes with AFL++
  12. Metamorphic oracles
  13. Motivating example
  14. A "hard-coded credentials" backdoor in sudo
  15. Detecting the backdoor with Rosa
  16. ...and 23 more sections

Figures (2)

  • Figure 1: General overview of the Rosa approach to fuzzing-based backdoor detection.
  • Figure 2: Rosa parameter sweep study for the duration of phase 1. For each duration, we performed a total of 180 runs of 8 hours (10 runs per backdoor in Rosarum, including with specialized seeds for the Belkin backdoor).