Reinforcement Learning of Large Language Models for Interpretable Credit Card Fraud Detection

TL;DR

This work tackles the difficulty of detecting credit card fraud in real-world e-commerce by fine-tuning lightweight LLMs through reinforcement learning. Using the GSPO framework with a rule-based reward, the models learn to extract textual trust and risk signals from raw transaction data, resulting in substantial improvements in $F1$-Score on held-out data while achieving concise, low-latency reasoning. A key finding is the strong benefit of exploration: open-ended standard training markedly outperforms compressed, signal-fixed prompts, and smaller models can surpass larger ones in task-specific calibration. The study also reveals a calibration gap for general-purpose LLMs in this domain and highlights challenges around faithfulness and reward design, motivating future work on faithfulness-aware objectives and hybrid systems that combine LLM interpretability with traditional models’ speed for structured features.

Abstract

E-commerce platforms and payment solution providers face increasingly sophisticated fraud schemes, ranging from identity theft and account takeovers to complex money laundering operations that exploit the speed and anonymity of digital transactions. However, despite their theoretical promise, the application of Large Language Models (LLMs) to fraud detection in real-world financial contexts remains largely unexploited, and their practical effectiveness in handling domain-specific e-commerce transaction data has yet to be empirically validated. To bridge this gap between conventional machine learning limitations and the untapped potential of LLMs in fraud detection, this paper proposes a novel approach that employs Reinforcement Learning (RL) to post-train lightweight language models specifically for fraud detection tasks using only raw transaction data. We utilize the Group Sequence Policy Optimization (GSPO) algorithm combined with a rule-based reward system to fine-tune language models of various sizes on a real-life transaction dataset provided by a Chinese global payment solution company. Through this reinforcement learning framework, the language models are encouraged to explore diverse trust and risk signals embedded within the textual transaction data, including patterns in customer information, shipping details, product descriptions, and order history. Our experimental results demonstrate the effectiveness of this approach, with post-trained language models achieving substantial F1-score improvements on held-out test data. Our findings demonstrate that the observed performance improvements are primarily attributable to the exploration mechanism inherent in reinforcement learning, which allows models to discover novel fraud indicators beyond those captured by traditional engineered features.

Figures (9)

• Figure 1: Overview of the GSPO reinforcement learning framework for fraud detection. For each input transaction, the policy model samples a group of candidate responses (rollouts). These outputs are evaluated against the ground truth label to compute rewards and derive group-relative advantage estimates. Finally, these advantages drive the GSPO optimization objective, updating the model parameters to reinforce accurate reasoning and verdicts. Note that the "Order Info" panel displays a representative subset of the complete feature set for brevity.
• Figure 2: Schematic of the reinforcement learning cycle for fraud detection. The forward path illustrates the model's reasoning chain: extracting diverse trust and risk signals from raw inputs and synthesizing evidence against a latent threshold to render a verdict. The backward path demonstrates optimization: reward signals derived from ground truth labels drive backpropagation, updating parameters to implicitly learn signal relevance and calibrate domain-specific decision boundaries.
• Figure 3: Prompt template for our fraud detection LMs.
• Figure 4: An example output generated by the trained Qwen3-14B on the test set.
• Figure 5: The prompt for the compressed setup and a response example generated by Qwen3-14B. The model is provided with a series of pre-defined risk and trust signals based on the prior experience of our anti-fraud experts, and is instructed to produce a determined, concise response. The format requirement and some defined signals are omitted in the figure for brevity.