Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing Training

Daniele Lain, Tarek Jost, Sinisa Matetic, Kari Kostiainen, Srdjan Capkun

TL;DR

Embedded phishing training is investigated in three aspects, and its effectiveness comes from its nudging effect, i.e., the periodic reminder of the threat rather than from its content, which is rarely consumed by employees due to lack of time and perceived usefulness.

Abstract

A common form of phishing training in organizations is the use of simulated phishing emails to test employees' susceptibility to phishing attacks, and the immediate delivery of training material to those who fail the test. This widespread practice is dubbed embedded training; however, its effectiveness in decreasing the likelihood of employees falling for phishing again in the future is questioned by the contradictory findings of several recent field studies. We investigate embedded phishing training in three aspects. First, we observe that the practice incorporates different components -- knowledge gains from its content, nudges and reminders from the test itself, and the deterrent effect of potential consequences -- our goal is to study which ones are more effective, if any. Second, we explore two potential improvements to training, namely its timing and the use of incentives. Third, we analyze employees' reception and perception of the practice. For this, we conducted a large-scale mixed-methods (quantitative and qualitative) study on the employees of a partner company. Our study contributes several novel findings on the training practice: in particular, its effectiveness comes from its nudging effect, i.e., the periodic reminder of the threat rather than from its content, which is rarely consumed by employees due to lack of time and perceived usefulness. Further, delaying training to ease time pressure is as effective as currently established practices, while rewards do not improve secure behavior. Finally, some of our results support previous findings with increased ecological validity, e.g., that phishing is an attention problem, rather than a knowledge one, even for the most susceptible employees, and thus enforcing training does not help.

Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing Training

TL;DR

Embedded phishing training is investigated in three aspects, and its effectiveness comes from its nudging effect, i.e., the periodic reminder of the threat rather than from its content, which is rarely consumed by employees due to lack of time and perceived usefulness.

Abstract

A common form of phishing training in organizations is the use of simulated phishing emails to test employees' susceptibility to phishing attacks, and the immediate delivery of training material to those who fail the test. This widespread practice is dubbed embedded training; however, its effectiveness in decreasing the likelihood of employees falling for phishing again in the future is questioned by the contradictory findings of several recent field studies. We investigate embedded phishing training in three aspects. First, we observe that the practice incorporates different components -- knowledge gains from its content, nudges and reminders from the test itself, and the deterrent effect of potential consequences -- our goal is to study which ones are more effective, if any. Second, we explore two potential improvements to training, namely its timing and the use of incentives. Third, we analyze employees' reception and perception of the practice. For this, we conducted a large-scale mixed-methods (quantitative and qualitative) study on the employees of a partner company. Our study contributes several novel findings on the training practice: in particular, its effectiveness comes from its nudging effect, i.e., the periodic reminder of the threat rather than from its content, which is rarely consumed by employees due to lack of time and perceived usefulness. Further, delaying training to ease time pressure is as effective as currently established practices, while rewards do not improve secure behavior. Finally, some of our results support previous findings with increased ecological validity, e.g., that phishing is an attention problem, rather than a knowledge one, even for the most susceptible employees, and thus enforcing training does not help.
Paper Structure (77 sections, 7 figures, 5 tables)

This paper contains 77 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Our study
  3. Outline
  4. Background and Related Work
  5. Phishing Training
  6. Embedded Training
  7. Training Effectiveness
  8. Research Questions & Main Findings
  9. Training Effectiveness
  10. Training Improvements
  11. Training timing
  12. Incentives
  13. Training Reception
  14. Summary of Main Findings
  15. Training Effectiveness
  16. ...and 62 more sections

Figures (7)

  • Figure 1: Experimental setup in the partner company infrastructure.
  • Figure 2: Dangerous actions for the embedded training groups. We show group 3 who only received embedded training, all groups receiving embedded training regardless of other conditions, and the control group.
  • Figure 3: Dangerous actions for the deterrent groups. We show group 8 who only received our deterrent, all groups receiving deterrents regardless of other conditions, and the control group.
  • Figure 4: Dangerous actions for the embedded training and deterrent groups. We show group 3 who only received embedded training, group 8 who only received our deterrent, group 2 who received both, and the control group.
  • Figure 5: Dangerous action by type of training. We show group 3, who only received immediate training, all other groups receiving immediate training, group 6, who only received delayed training, and all other groups receiving delayed training.
  • ...and 2 more figures