Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KVC-onGoing: Keystroke Verification Challenge

Giuseppe Stragapede, Ruben Vera-Rodriguez, Ruben Tolosana, Aythami Morales, Ivan DeAndres-Tame, Naser Damer, Julian Fierrez, Javier Ortega-Garcia, Alejandro Acien, Nahuel Gonzalez, Andrei Shadrikov, Dmitrii Gordin, Leon Schmitt, Daniel Wimmer, Christoph Großmann, Joerdis Krieger, Florian Heinz, Ron Krestel, Christoffer Mayer, Simon Haberl, Helena Gschrey, Yosuke Yamagishi, Sanjay Saha, Sanka Rasnayaka, Sandareka Wickramanayake, Terence Sim, Weronika Gutfeter, Adam Baran, Mateusz Krzysztoń, Przemysław Jaskóła

TL;DR

KVC-onGoing introduces a large-scale, open benchmark for keystroke dynamics-based biometric verification, unifying data from the Aalto Desktop and Mobile Keystroke Databases under a standardized experimental protocol hosted on CodaLab. The framework supports desktop and mobile tasks with open-set learning, extensive enrolment/verification schedules, and dual evaluation modes to assess global and per-subject performance, paired with a comprehensive fairness analysis. Across diverse architectures (RNNs, CNNs, Transformers) and losses (SetMargin, ArcFace, triplet, contrastive), the study identifies LSIA and VeriKVC as leading solutions, achieving $EER$ as low as $3.33$% (desktop) and $3.61$% (mobile), demonstrating the strong discriminative power of keystroke dynamics. The work also highlights modest demographic biases in KD and emphasizes the need for further investigation into fairness across age and gender, suggesting future directions in architecture optimization and richer demographic analyses to strengthen practical deployment and privacy considerations.

Abstract

This article presents the Keystroke Verification Challenge - onGoing (KVC-onGoing), on which researchers can easily benchmark their systems in a common platform using large-scale public databases, the Aalto University Keystroke databases, and a standard experimental protocol. The keystroke data consist of tweet-long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards simulating real-life conditions. The results on the evaluation set of KVC-onGoing have proved the high discriminative power of keystroke dynamics, reaching values as low as 3.33% of Equal Error Rate (EER) and 11.96% of False Non-Match Rate (FNMR) @1% False Match Rate (FMR) in the desktop scenario, and 3.61% of EER and 17.44% of FNMR @1% at FMR in the mobile scenario, significantly improving previous state-of-the-art results. Concerning demographic fairness, the analyzed scores reflect the subjects' age and gender to various extents, not negligible in a few cases. The framework runs on CodaLab.

KVC-onGoing: Keystroke Verification Challenge

TL;DR

KVC-onGoing introduces a large-scale, open benchmark for keystroke dynamics-based biometric verification, unifying data from the Aalto Desktop and Mobile Keystroke Databases under a standardized experimental protocol hosted on CodaLab. The framework supports desktop and mobile tasks with open-set learning, extensive enrolment/verification schedules, and dual evaluation modes to assess global and per-subject performance, paired with a comprehensive fairness analysis. Across diverse architectures (RNNs, CNNs, Transformers) and losses (SetMargin, ArcFace, triplet, contrastive), the study identifies LSIA and VeriKVC as leading solutions, achieving as low as % (desktop) and % (mobile), demonstrating the strong discriminative power of keystroke dynamics. The work also highlights modest demographic biases in KD and emphasizes the need for further investigation into fairness across age and gender, suggesting future directions in architecture optimization and richer demographic analyses to strengthen practical deployment and privacy considerations.

Abstract

This article presents the Keystroke Verification Challenge - onGoing (KVC-onGoing), on which researchers can easily benchmark their systems in a common platform using large-scale public databases, the Aalto University Keystroke databases, and a standard experimental protocol. The keystroke data consist of tweet-long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards simulating real-life conditions. The results on the evaluation set of KVC-onGoing have proved the high discriminative power of keystroke dynamics, reaching values as low as 3.33% of Equal Error Rate (EER) and 11.96% of False Non-Match Rate (FNMR) @1% False Match Rate (FMR) in the desktop scenario, and 3.61% of EER and 17.44% of FNMR @1% at FMR in the mobile scenario, significantly improving previous state-of-the-art results. Concerning demographic fairness, the analyzed scores reflect the subjects' age and gender to various extents, not negligible in a few cases. The framework runs on CodaLab.
Paper Structure (35 sections, 1 equation, 3 figures, 6 tables)

This paper contains 35 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Resources Provided
  4. Experimental Protocol
  5. Metrics Adopted
  6. Biometric Verification Systems
  7. Team: LSIA
  8. Model Architecture
  9. Training
  10. Team: VeriKVC
  11. Model Architecture
  12. Training
  13. Team: Keystroke Wizards
  14. Model Architecture
  15. Training
  16. ...and 20 more sections

Figures (3)

  • Figure 1: The dual-branch (recurrent and convolutional) embedding model for distance metric learning proposed by the LSIA team, ranked first in both tasks.
  • Figure 2: DET curves including the results of all the biometric verification systems proposed in the KVC-onGoing challenge for both desktop and mobile tasks. The red dashed lines indicate the operational points 0.1% FMR, 1% FMR and 10% FMR whereas the black dashed line indicate the points where the FMR = FNMR, corresponding to the EER.
  • Figure 3: Heat maps (normal and binarized) of the impostor scores of the YYama system with the highest $SIR_{a}$ (top) and $SIR_{g}$ (bottom) (YYama).