Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Explain, Don't Just Warn!" -- A Real-Time Framework for Generating Phishing Warnings with Contextual Cues

Sayak Saha Roy, Cesar Torres, Shirin Nilizadeh

TL;DR

PhishXplain introduces a real-time, on-device framework that replaces generic phishing warnings with contextual explanations and annotated visuals. By leveraging a lightweight, quantized LLM (LLaMA 3.2:3B) and a two-prompt lookup-table workflow, it detects user-facing phishing cues from client-side source code and presents explanations with bounding-box highlights. Across a longitudinal real-world deployment and a controlled user study, PXP improves phishing detection accuracy, user confidence, and perceived usefulness, especially for users with lower cybersecurity proficiency, while preserving privacy through local inference. The work demonstrates practical viability on consumer hardware and provides a browser-extension implementation to encourage adoption. Overall, PhishXplain advances user education and resilience against phishing by making warnings actionable rather than merely prescriptive.

Abstract

Anti-phishing tools typically display generic warnings that offer users limited explanation on why a website is considered malicious, which can prevent end-users from developing the mental models needed to recognize phishing cues on their own. This becomes especially problematic when these tools inevitably fail - particularly against evasive threats, and users are found to be ill-equipped to identify and avoid them independently. To address these limitations, we present PhishXplain (PXP), a real-time explainable phishing warning system designed to augment existing detection mechanisms. PXP empowers users by clearly articulating why a site is flagged as malicious, highlighting suspicious elements using a memory-efficient implementation of LLaMA 3.2. It utilizes a structured two-step prompt architecture to identify phishing features, generate contextual explanations, and render annotated screenshots that visually reinforce the warning. Longitudinally implementing PhishXplain over a month on 7,091 live phishing websites, we found that it can generate warnings for 94% of the sites, with a correctness of 96%. We also evaluated PhishXplain through a user study with 150 participants split into two groups: one received conventional, generic warnings, while the other interacted with PXP's explainable alerts. Participants who received the explainable warnings not only demonstrated a significantly better understanding of phishing indicators but also achieved higher accuracy in identifying phishing threats, even without any warning. Moreover, they reported greater satisfaction and trust in the warnings themselves. These improvements were especially pronounced among users with lower initial levels of cybersecurity proficiency and awareness. To encourage the adoption of this framework, we release PhishXplain as a browser extension.

"Explain, Don't Just Warn!" -- A Real-Time Framework for Generating Phishing Warnings with Contextual Cues

TL;DR

PhishXplain introduces a real-time, on-device framework that replaces generic phishing warnings with contextual explanations and annotated visuals. By leveraging a lightweight, quantized LLM (LLaMA 3.2:3B) and a two-prompt lookup-table workflow, it detects user-facing phishing cues from client-side source code and presents explanations with bounding-box highlights. Across a longitudinal real-world deployment and a controlled user study, PXP improves phishing detection accuracy, user confidence, and perceived usefulness, especially for users with lower cybersecurity proficiency, while preserving privacy through local inference. The work demonstrates practical viability on consumer hardware and provides a browser-extension implementation to encourage adoption. Overall, PhishXplain advances user education and resilience against phishing by making warnings actionable rather than merely prescriptive.

Abstract

Anti-phishing tools typically display generic warnings that offer users limited explanation on why a website is considered malicious, which can prevent end-users from developing the mental models needed to recognize phishing cues on their own. This becomes especially problematic when these tools inevitably fail - particularly against evasive threats, and users are found to be ill-equipped to identify and avoid them independently. To address these limitations, we present PhishXplain (PXP), a real-time explainable phishing warning system designed to augment existing detection mechanisms. PXP empowers users by clearly articulating why a site is flagged as malicious, highlighting suspicious elements using a memory-efficient implementation of LLaMA 3.2. It utilizes a structured two-step prompt architecture to identify phishing features, generate contextual explanations, and render annotated screenshots that visually reinforce the warning. Longitudinally implementing PhishXplain over a month on 7,091 live phishing websites, we found that it can generate warnings for 94% of the sites, with a correctness of 96%. We also evaluated PhishXplain through a user study with 150 participants split into two groups: one received conventional, generic warnings, while the other interacted with PXP's explainable alerts. Participants who received the explainable warnings not only demonstrated a significantly better understanding of phishing indicators but also achieved higher accuracy in identifying phishing threats, even without any warning. Moreover, they reported greater satisfaction and trust in the warnings themselves. These improvements were especially pronounced among users with lower initial levels of cybersecurity proficiency and awareness. To encourage the adoption of this framework, we release PhishXplain as a browser extension.
Paper Structure (20 sections, 3 equations, 20 figures, 6 tables)

This paper contains 20 sections, 3 equations, 20 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Framework
  4. URL and Source Code Capture
  5. Parsing the source-code
  6. Feature Extraction
  7. Generating the warning
  8. Client-side application
  9. Evaluating our framework
  10. Speed and memory costs
  11. Reliability Evaluation and Scoring Metric
  12. Real-world feasibility
  13. User Study
  14. Research Questions and Hypotheses
  15. Study Components
  16. ...and 5 more sections

Figures (20)

  • Figure 1: The generic phishing warning shown by Google Safe Browsing with no contextual information
  • Figure 2: The PhishXplain framework
  • Figure 3: Example of PhishXplain's explainable blocklisting warning page showcasing contextual details about suspicious features present in the detected Facebook phishing page.
  • Figure 4: A <p> tag encapsulated by an identifier. Note the URL is appended at the start as ELEMENT 0. The URL is visible since it is a placeholder URL.
  • Figure 5: The first prompt to extract features from the source code
  • ...and 15 more figures