FraudFox: Adaptable Fraud Detection in the Real World

Abstract

The proposed method (FraudFox) provides solutions to adversarial attacks in a resource constrained environment. We focus on questions like the following: How suspicious is `Smith', trying to buy \$500 shoes, on Monday 3am? How to merge the risk scores, from a handful of risk-assessment modules (`oracles') in an adversarial environment? More importantly, given historical data (orders, prices, and what-happened afterwards), and business goals/restrictions, which transactions, like the `Smith' transaction above, which ones should we `pass', versus send to human investigators? The business restrictions could be: `at most $x$ investigations are feasible', or `at most \$$y$ lost due to fraud'. These are the two research problems we focus on, in this work. One approach to address the first problem (`oracle-weighting'), is by using Extended Kalman Filters with dynamic importance weights, to automatically and continuously update our weights for each 'oracle'. For the second problem, we show how to derive an optimal decision surface, and how to compute the Pareto optimal set, to allow what-if questions. An important consideration is adaptation: Fraudsters will change their behavior, according to our past decisions; thus, we need to adapt accordingly. The resulting system, \method, is scalable, adaptable to changing fraudster behavior, effective, and already in \textbf{production} at Amazon. FraudFox augments a fraud prevention sub-system and has led to significant performance gains.

Figures (5)

• Figure 1: Business Impact of FraudFox: Visible loss reduction when applying it to real word data. Losses vs. time - 1st without dynamically blending a subset of fraud indicators (blue), i.e., without FraudFox; 2nd with dynamically blending (orange), i.e. with FraudFox introduced at the dashed-black time tick.
• Figure 2: Illustration of Decision Surface and Adversarial Behavior. In the (fraud-score vs. order value) plot, when we fix the decision surface (blue curve), fraudulent orders (red crosses) will probably flock just below it. FraudFox-I is blocking exactly this behavior.
• Figure 3: Cost Benefit Analysis. Profit, for each of the four cases.
• Figure 4: Business Adaptation and Pareto Front.FraudFox precomputes the Pareto front, in preparation of changes in business requirements. Scatter plot of model performance in business metric space (# of investigations, vs $ fraud captured per investigation). Every point is a possible parameter choice of FraudFox. Points with empty light blue NE quadrants are non-dominating and thus form the Pareto front (green 'x' points).
• Figure 5: FraudFox Adapts: Synthetic Example. As expected, both our versions (equal-weights, in 'green' and exponentially-decaying weights, in 'red') responds to a change/shock at $t$=30, and eventually recover from the shock. Without adaptation ('blue'), the behavior suffers.

Theorems & Definitions (10)

• Lemma 1: EKF update
• proof
• theorem 1: EKF with forgetting
• Proof 1
• Lemma 2
• Proof 2
• Lemma 3
• Proof 3
• Lemma 4: Hyperbolic decision surface
• proof