REVERSIM: An Open-Source Environment for the Controlled Study of Human Aspects in Hardware Reverse Engineering

TL;DR

Re ReverSim addresses the challenge of studying human factors in hardware reverse engineering by delivering an open-source, web-based environment that standardizes HRE subproblems and integrates standardized cognitive tests. The authors validate ReverSim with expert feedback and large non-expert samples, demonstrating that it captures key HRE aspects, differentiates performance across task difficulties, and reveals correlations between cognitive processing speed and task performance. This platform enables scalable, controlled studies of cognition in HRE with potential implications for cognitive obfuscation strategies and hardware-protection design, as well as education and skill assessment. Limitations include simplified task complexity and fatigue effects in online cognitive testing, with future work to include sequential components, higher-level gates, and broader psychometric batteries to enrich insights into human aspects of HRE.

Abstract

Hardware Reverse Engineering (HRE) is a technique for analyzing integrated circuits. Experts employ HRE for security-critical tasks, like detecting Trojans or intellectual property violations, relying not only on their experience and customized tools but also on their cognitive abilities. In this work, we introduce ReverSim, a software environment that models key HRE subprocesses and integrates standardized cognitive tests. ReverSim enables quantitative studies with easier-to-recruit non-experts to uncover cognitive factors relevant to HRE. We empirically evaluated ReverSim in three studies. Semi-structured interviews with 14 HRE professionals confirmed its comparability to real-world HRE processes. Two online user studies with 170 novices and intermediates revealed effective differentiation of participant performance across a spectrum of difficulties, and correlations between participants' cognitive processing speed and task performance. ReverSim is available as open-source software, providing a robust platform for controlled experiments to assess cognitive processes in HRE, potentially opening new avenues for hardware protection.

Figures (20)

• Figure 1: Explanation of the typical phases when using ReverSim for controlled studies.
• Figure 2: Sketch of the development process of ReverSim in terms of testing and deployment (top) as well as major development steps and examples of continuous improvement (bottom). We implemented a full, working prototype of ReverSim, followed by a round of internal piloting and feedback from hardware security researchers, e. g., resulting in the addition of the interactive tutorial. Piloting of the resulting base environment took place during an in-person academic workshop with a separate group of HCI and security researchers and professionals, e. g., prompting the introduction of the drawing tools. After conducting the interview study (see \ref{['section:interview']}), we further revised ReverSim and piloted our study setup in two rounds, each with independently recruited participants from industry and academia. We performed a final piloting round with participants from Prolific. Based on the finalized study setup, we conducted two user studies with novices (see \ref{['section:pilot']}) and intermediates (see \ref{['appendix:intermediatesample']}).
• Figure 3: Lamp and danger sign indicate the expected output of a circuit. Participants solve the task by supplying current to the lamp, turning it on (bottom right), and by ensuring that no current is supplied to the danger sign (top left).
• Figure 4: The interface for each task of ReverSim consists of a Boolean circuit diagram with three inputs and at least one output. The participant interacts with the circuit by opening and closing the switches on the left. The example level shown here consists of three switches, three gates (AND, OR, and NOT) and one lamp as an output. Annotations can be drawn onto the circuit using the drawing tools on the very left. At the top, the participant's progress statistics are displayed.
• Figure 5: The visualization of a camouflaged and a covert gate (left) and an example of their hidden functionality (right).