Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Swap It Like Its Hot: Segmentation-based spoof attacks on eye-tracking images

Anish S. Narkar, Brendan David-John

TL;DR

This work exposes a new vulnerability in gaze-based iris authentication by introducing IrisSwap, a pipeline that segments an attacker’s eye image and digitally swaps in a victim’s iris texture to defeat liveness detection while preserving gaze signals. By leveraging a shallow-double-UNET iris segmentation (Shallow-net), an inverse rubber sheet-based iris texture swap, and a Pupil Labs gaze-estimation framework, the authors demonstrate offline and online attacks that bypass state-of-the-art liveness detectors with user-level attack success around $0.58$ and $0.55$ respectively. Iris authentication remains exploitable under these conditions, with mean iris-authentication HD around $0.36$–$0.39$ and a substantial portion of victims impersonated, highlighting a pressing need for defenses beyond velocity-based liveness. The findings motivate the development of robust iris-authentication schemes and artifact-based, peri-ocular, or watermarking defenses to counter digital iris manipulation in eye-tracking systems, and they underscore practical security concerns for deploying gaze-based biometric authentication in real-world devices.

Abstract

Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which is sufficient to detect physical presentation attacks. However, such defenses cannot detect a spoofing attack when real eye image inputs are digitally manipulated to swap the iris pattern of another person. We propose IrisSwap as a novel attack on gaze-based liveness detection. IrisSwap allows attackers to segment and digitally swap in a victim's iris pattern to fool iris authentication. Both offline and online attacks produce gaze data that deceives the current state-of-the-art defense models at rates up to 58% and motivates the need to develop more advanced authentication methods for eye trackers.

Swap It Like Its Hot: Segmentation-based spoof attacks on eye-tracking images

TL;DR

This work exposes a new vulnerability in gaze-based iris authentication by introducing IrisSwap, a pipeline that segments an attacker’s eye image and digitally swaps in a victim’s iris texture to defeat liveness detection while preserving gaze signals. By leveraging a shallow-double-UNET iris segmentation (Shallow-net), an inverse rubber sheet-based iris texture swap, and a Pupil Labs gaze-estimation framework, the authors demonstrate offline and online attacks that bypass state-of-the-art liveness detectors with user-level attack success around and respectively. Iris authentication remains exploitable under these conditions, with mean iris-authentication HD around – and a substantial portion of victims impersonated, highlighting a pressing need for defenses beyond velocity-based liveness. The findings motivate the development of robust iris-authentication schemes and artifact-based, peri-ocular, or watermarking defenses to counter digital iris manipulation in eye-tracking systems, and they underscore practical security concerns for deploying gaze-based biometric authentication in real-world devices.

Abstract

Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which is sufficient to detect physical presentation attacks. However, such defenses cannot detect a spoofing attack when real eye image inputs are digitally manipulated to swap the iris pattern of another person. We propose IrisSwap as a novel attack on gaze-based liveness detection. IrisSwap allows attackers to segment and digitally swap in a victim's iris pattern to fool iris authentication. Both offline and online attacks produce gaze data that deceives the current state-of-the-art defense models at rates up to 58% and motivates the need to develop more advanced authentication methods for eye trackers.
Paper Structure (34 sections, 2 equations, 4 figures, 1 table)

This paper contains 34 sections, 2 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Attack Model And Methodology
  4. Design Challenges
  5. Challenge 1: Accurate segmentation and efficient swapping
  6. Challenge 2: Minimal impact on gaze estimation
  7. IrisSwap System Design
  8. Iris Segmentation
  9. Iris Texture Swap
  10. Gaze Estimation
  11. Challenge Task
  12. System Deployment
  13. Datasets and Experiments
  14. Datasets
  15. Offline Attack Dataset
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Illustration of the spoofing attack pipeline for iris patterns with gaze-based liveness detection.
  • Figure 2: IrisSwap pipeline showing the flow of data through iris segmentation and swapping before gaze estimation is applied. The output gaze positions are processed into windows of gaze velocity that are classified by a liveness detection model.
  • Figure 3: Velocity profiles for Unswapped (Left) and Swapped (Right) samples over five seconds of an online attack. A similar profile is produced between 5s and 7s while the differences between 7s and 9s are flagged as spoofs by the liveness model.
  • Figure 4: Window-level ASR indicates the success rate calculated based on each window of the test-user data. Window-level ASR is 0.61 $\pm$ 0.07 and 0.58 $\pm$ 0.08 for offline and online attacks, respectively. User-level ASR makes one real or spoof classification based on all windows from a single user. User-level ASR is 0.59 $\pm$ 0.10 and 0.55 $\pm$ 0.06 for offline and online attacks, respectively.