Named Entity Recognition for Payment Data Using NLP

TL;DR

This work addresses the challenge of extracting structured entities from diverse payment messages by framing NER as a domain-specific sequence labeling task. It systematically benchmarks classical CRF with domain features, BiLSTM-CRF, and transformer-based models, introducing PaymentBERT, a hybrid architecture that fuses contextual BERT representations with payment-aware embeddings and format features. On a 50,000-message dataset spanning SWIFT, ISO 20022, and domestic formats, PaymentBERT achieves 95.7% F1-score, with robust cross-format generalization and favorable latency characteristics; distilled and quantized variants offer practical throughput improvements. The study provides extensive ablations, error analyses, and deployment guidance, demonstrating that transformer-based models augmented with domain knowledge can meet production requirements for sanctions screening, AML compliance, and payment processing. The findings have direct implications for financial institutions seeking accurate, scalable NER for automated transaction monitoring and processing systems.

Abstract

Named Entity Recognition (NER) has emerged as a critical component in automating financial transaction processing, particularly in extracting structured information from unstructured payment data. This paper presents a comprehensive analysis of state-of-the-art NER algorithms specifically designed for payment data extraction, including Conditional Random Fields (CRF), Bidirectional Long Short-Term Memory with CRF (BiLSTM-CRF), and transformer-based models such as BERT and FinBERT. We conduct extensive experiments on a dataset of 50,000 annotated payment transactions across multiple payment formats including SWIFT MT103, ISO 20022, and domestic payment systems. Our experimental results demonstrate that fine-tuned BERT models achieve an F1-score of 94.2% for entity extraction, outperforming traditional CRF-based approaches by 12.8 percentage points. Furthermore, we introduce PaymentBERT, a novel hybrid architecture combining domain-specific financial embeddings with contextual representations, achieving state-of-the-art performance with 95.7% F1-score while maintaining real-time processing capabilities. We provide detailed analysis of cross-format generalization, ablation studies, and deployment considerations. This research provides practical insights for financial institutions implementing automated sanctions screening, anti-money laundering (AML) compliance, and payment processing systems.

Figures (8)

• Figure 1: Performance comparison across different NER models. PaymentBERT achieves the highest scores across all metrics while maintaining reasonable inference latency.
• Figure 2: Per-entity-type F1-scores across different models. PERSON_NAME and AMOUNT entities are most accurately extracted, while PURPOSE entities pose the greatest challenge.
• Figure 3: Cross-format generalization performance. PaymentBERT maintains more consistent performance across different payment message formats compared to other models.
• Figure 4: Ablation study showing contribution of each component in PaymentBERT. Removing payment embeddings or format features both result in notable performance degradation.
• Figure 5: Training and validation loss curves for BiLSTM-CRF, BERT-base, and PaymentBERT. PaymentBERT converges faster and achieves lower validation loss.