Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Impact of Data Breadth and Depth on Performance of Siamese Neural Network Model: Experiments with Three Keystroke Dynamic Datasets

Ahmed Anu Wahab, Daqing Hou, Nadia Cheng, Parker Huntley, Charles Devlen

TL;DR

The paper addresses how data breadth (subject count) and depth (per-subject data) affect Siamese neural networks for keystroke dynamics authentication. It employs a unified TypeNet-like SNN, experiments on three public datasets with controlled breadth/depth variation, and analyzes results through the lens of feature space and density. Key findings show breadth markedly boosts generalization, especially in free-text datasets, while depth effects are dataset-dependent; fixed-text data saturates quickly, whereas free-text data benefits from more varied data and longer sequences. The work provides actionable guidance for constructing scalable keystroke biometric systems and highlights the need for larger, more diverse datasets to push the state of the art.

Abstract

Deep learning models, such as the Siamese Neural Networks (SNN), have shown great potential in capturing the intricate patterns in behavioral data. However, the impacts of dataset breadth (i.e., the number of subjects) and depth (e.g., the amount of training samples per subject) on the performance of these models is often informally assumed, and remains under-explored. To this end, we have conducted extensive experiments using the concepts of "feature space" and "density" to guide and gain deeper understanding on the impact of dataset breadth and depth on three publicly available keystroke datasets (Aalto, CMU and Clarkson II). Through varying the number of training subjects, number of samples per subject, amount of data in each sample, and number of triplets used in training, we found that when feasible, increasing dataset breadth enables the training of a well-trained model that effectively captures more inter-subject variability. In contrast, we find that the extent of depth's impact from a dataset depends on the nature of the dataset. Free-text datasets are influenced by all three depth-wise factors; inadequate samples per subject, sequence length, training triplets and gallery sample size, which may all lead to an under-trained model. Fixed-text datasets are less affected by these factors, and as such make it easier to create a well-trained model. These findings shed light on the importance of dataset breadth and depth in training deep learning models for behavioral biometrics and provide valuable insights for designing more effective authentication systems.

Impact of Data Breadth and Depth on Performance of Siamese Neural Network Model: Experiments with Three Keystroke Dynamic Datasets

TL;DR

The paper addresses how data breadth (subject count) and depth (per-subject data) affect Siamese neural networks for keystroke dynamics authentication. It employs a unified TypeNet-like SNN, experiments on three public datasets with controlled breadth/depth variation, and analyzes results through the lens of feature space and density. Key findings show breadth markedly boosts generalization, especially in free-text datasets, while depth effects are dataset-dependent; fixed-text data saturates quickly, whereas free-text data benefits from more varied data and longer sequences. The work provides actionable guidance for constructing scalable keystroke biometric systems and highlights the need for larger, more diverse datasets to push the state of the art.

Abstract

Deep learning models, such as the Siamese Neural Networks (SNN), have shown great potential in capturing the intricate patterns in behavioral data. However, the impacts of dataset breadth (i.e., the number of subjects) and depth (e.g., the amount of training samples per subject) on the performance of these models is often informally assumed, and remains under-explored. To this end, we have conducted extensive experiments using the concepts of "feature space" and "density" to guide and gain deeper understanding on the impact of dataset breadth and depth on three publicly available keystroke datasets (Aalto, CMU and Clarkson II). Through varying the number of training subjects, number of samples per subject, amount of data in each sample, and number of triplets used in training, we found that when feasible, increasing dataset breadth enables the training of a well-trained model that effectively captures more inter-subject variability. In contrast, we find that the extent of depth's impact from a dataset depends on the nature of the dataset. Free-text datasets are influenced by all three depth-wise factors; inadequate samples per subject, sequence length, training triplets and gallery sample size, which may all lead to an under-trained model. Fixed-text datasets are less affected by these factors, and as such make it easier to create a well-trained model. These findings shed light on the importance of dataset breadth and depth in training deep learning models for behavioral biometrics and provide valuable insights for designing more effective authentication systems.
Paper Structure (23 sections, 3 equations, 4 figures, 8 tables)

This paper contains 23 sections, 3 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Siamese Neural Network Architecture
  4. Datasets
  5. The Aalto Dataset
  6. The CMU Dataset
  7. The Clarkson II Dataset
  8. Data Preprocessing
  9. Experimental Procedures and Results
  10. Breadth-wise Experiments
  11. Depth-wise Experiments
  12. Varying Number of Samples per Subject
  13. Varying Sample Sequence Length
  14. Varying Amount of Training Triplets
  15. Model Evaluation
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Quadrant plots of datasets based on breadth and depth
  • Figure 2: (a) The Siamese sub-network, taking a time series input (x$i$) of shape $m\times n$ and returning an output vector (embeddings) of shape $1\times 128$. (b) The Siamese network, consisting of three (3) sub-networks. Loss is calculated from the three output vectors and are back-propagated into the network.
  • Figure 3: A screenshot of the preprocessed CMU dataset highlighting its four distinctive features (m, ud, dd, and id).
  • Figure 4: Aalto Dataset: Box plots for both the breadth-wise experiments (as seen horizontally with varying number of subjects), and the depth-wise experiments (as seen vertically with varying number of samples per subject) for 7.6 million triplets, where $M=70$. Each box plot displays the EERs from ten reruns.