Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fin-Fed-OD: Federated Outlier Detection on Financial Tabular Data

Dayananda Herurkar, Sebastian Palacio, Ahmed Anwar, Joern Hees, Andreas Dengel

TL;DR

Fin-Fed-OD tackles open-world anomaly detection in privacy-sensitive financial tabular data by marrying representation learning with federated learning. The method trains client-specific autoencoders to learn latent representations, which are then used to refine inlier boundaries via OD models while sharing only model parameters through FL aggregation (FedAvg/FedProx). Across tabular and image datasets, including DAGMM and MemAE baselines, FL-based approaches yield stronger detection of unknown outliers without degrading known-outlier performance, supported by both quantitative AP improvements and qualitative latent-space clustering. This privacy-preserving collaboration enables robust, client-discriminating OD with practical implications for financial fraud detection and cross-organization anomaly resilience.

Abstract

Anomaly detection in real-world scenarios poses challenges due to dynamic and often unknown anomaly distributions, requiring robust methods that operate under an open-world assumption. This challenge is exacerbated in practical settings, where models are employed by private organizations, precluding data sharing due to privacy and competitive concerns. Despite potential benefits, the sharing of anomaly information across organizations is restricted. This paper addresses the question of enhancing outlier detection within individual organizations without compromising data confidentiality. We propose a novel method leveraging representation learning and federated learning techniques to improve the detection of unknown anomalies. Specifically, our approach utilizes latent representations obtained from client-owned autoencoders to refine the decision boundary of inliers. Notably, only model parameters are shared between organizations, preserving data privacy. The efficacy of our proposed method is evaluated on two standard financial tabular datasets and an image dataset for anomaly detection in a distributed setting. The results demonstrate a strong improvement in the classification of unknown outliers during the inference phase for each organization's model.

Fin-Fed-OD: Federated Outlier Detection on Financial Tabular Data

TL;DR

Fin-Fed-OD tackles open-world anomaly detection in privacy-sensitive financial tabular data by marrying representation learning with federated learning. The method trains client-specific autoencoders to learn latent representations, which are then used to refine inlier boundaries via OD models while sharing only model parameters through FL aggregation (FedAvg/FedProx). Across tabular and image datasets, including DAGMM and MemAE baselines, FL-based approaches yield stronger detection of unknown outliers without degrading known-outlier performance, supported by both quantitative AP improvements and qualitative latent-space clustering. This privacy-preserving collaboration enables robust, client-discriminating OD with practical implications for financial fraud detection and cross-organization anomaly resilience.

Abstract

Anomaly detection in real-world scenarios poses challenges due to dynamic and often unknown anomaly distributions, requiring robust methods that operate under an open-world assumption. This challenge is exacerbated in practical settings, where models are employed by private organizations, precluding data sharing due to privacy and competitive concerns. Despite potential benefits, the sharing of anomaly information across organizations is restricted. This paper addresses the question of enhancing outlier detection within individual organizations without compromising data confidentiality. We propose a novel method leveraging representation learning and federated learning techniques to improve the detection of unknown anomalies. Specifically, our approach utilizes latent representations obtained from client-owned autoencoders to refine the decision boundary of inliers. Notably, only model parameters are shared between organizations, preserving data privacy. The efficacy of our proposed method is evaluated on two standard financial tabular datasets and an image dataset for anomaly detection in a distributed setting. The results demonstrate a strong improvement in the classification of unknown outliers during the inference phase for each organization's model.
Paper Structure (21 sections, 3 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 21 sections, 3 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. Standalone (Baseline) Model For Outlier Detection
  5. Collaborative Federated Learning Environment (CoFLE)
  6. Experimental Setup
  7. Tabular Dataset + Synthetic Outliers
  8. Image Dataset + Natural Outliers
  9. Model Parameters
  10. Evaluation Metric
  11. Results
  12. Quantitative Evaluation
  13. Qualitative Evaluation
  14. Conclusion and Future Work
  15. Quatitative Evaluation Result
  16. ...and 6 more sections

Figures (8)

  • Figure 1: Standalone models trained using local data (Inliers: Blue) favor detecting only known outliers (Bank1:Red, Bank2:Green, Bank3:Grey) but are not robust against unknown outliers from other organizations. The aggregated model can adapt to distinct representations and adjust its boundaries to detect unknown outliers.
  • Figure 2: FL-OD Setup
  • Figure 3: Results of OD models clientwise. FL-OD models (Fed_Avg+RF, Fed_Prox+RF) outperform the baseline (Baseline_RF) in detecting unknown outliers. Every score reflects the mean and standard deviation from five experiments.
  • Figure 4: Latent Space Visualization of Baseline, Fed_Avg, and Fed_Prox models
  • Figure 5: Results of OD models clientwise. FL-OD models (Fed_Avg+RF, Fed_Prox+RF) outperform the baseline (Baseline_RF) in detecting unknown outliers. Every score reflects the mean and standard deviation from five experiments.
  • ...and 3 more figures