Honeyquest: Rapidly Measuring the Enticingness of Cyber Deception Techniques with Code-based Questionnaires

TL;DR

Honeyquest tackles the reproducibility challenge in cyber deception by translating 25 CDTs (13 from prior work plus 12 self-defined) and 19 risks into HoneYAML specifications and evaluating them through a web-based questionnaire with 47 participants. By presenting attacker-like views as neutral, risky, or deceptive queries and collecting ordered marks on lines, the framework provides a rapid, sample-efficient measurement of CDT enticingness that aligns with prior CTF/honeypot findings while avoiding production deployments. The study reports that deception can reduce the likelihood of attackers exploiting true weaknesses by about $22\%$ on average, and it offers a scalable methodology and data pipeline for iterative CDT design and replication. Overall, Honeyquest yields actionable insights for CDT design and demonstrates a cost-effective path to empirically validate cyber deception before real-world deployment.

Abstract

Fooling adversaries with traps such as honeytokens can slow down cyber attacks and create strong indicators of compromise. Unfortunately, cyber deception techniques are often poorly specified. Also, realistically measuring their effectiveness requires a well-exposed software system together with a production-ready implementation of these techniques. This makes rapid prototyping challenging. Our work translates 13 previously researched and 12 self-defined techniques into a high-level, machine-readable specification. Our open-source tool, Honeyquest, allows researchers to quickly evaluate the enticingness of deception techniques without implementing them. We test the enticingness of 25 cyber deception techniques and 19 true security risks in an experiment with 47 humans. We successfully replicate the goals of previous work with many consistent findings, but without a time-consuming implementation of these techniques on real computer systems. We provide valuable insights for the design of enticing deception and also show that the presence of cyber deception can significantly reduce the risk that adversaries will find a true security risk by about 22% on average.

Figures (6)

• Figure 1: The lifecycle of designing, evaluating, and deploying CDTs, with the ultimate goal of engaging real adversaries.
• Figure 2: In Honeyquest, users are presented with *M neutral, *M risky, and *M deceptive queries. A line annotation set $L$ indicates the risky or deceptive lines in the associated query $q$. An answer marks vector $\mathbf{a}$ holds placed marks in order. The probabilistic algorithm $\mathrm{D}(q;~T)$ makes queries deceptive.
• Figure 3: Distribution of neutral, deceptive, and risky labels.
• Figure 4: Boxplot on the number of answered queries per participant. Total Queries = 174. Mean = 76. Median = 59.
• Figure 5: Our participants' self-reported profiles.