Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advanced Real-Time Fraud Detection Using RAG-Based LLMs

Gurjot Singh, Prabhjot Singh, Maninder Singh

TL;DR

This work tackles real-time fraud detection in phone communications by integrating Retrieval-Augmented Generation (RAG) with policy-driven reasoning. It presents a two-pronged system: real-time transcription with a policy-compliance check and a robust two-step user impersonation verification, all capable of updating policies without full model retraining. Evaluated on synthetic call data against BERT and untrained LLM baselines, the RAG approach achieves up to approximately 0.98 accuracy and establishes superior performance while remaining adaptable to evolving policies. The methodology promises practical deployment potential, combining up-to-date policy retrieval, transparent decision-making, and secure data handling to mitigate AI-enabled fraud in real-world scenarios.

Abstract

Artificial Intelligence has become a double edged sword in modern society being both a boon and a bane. While it empowers individuals it also enables malicious actors to perpetrate scams such as fraudulent phone calls and user impersonations. This growing threat necessitates a robust system to protect individuals In this paper we introduce a novel real time fraud detection mechanism using Retrieval Augmented Generation technology to address this challenge on two fronts. First our system incorporates a continuously updating policy checking feature that transcribes phone calls in real time and uses RAG based models to verify that the caller is not soliciting private information thus ensuring transparency and the authenticity of the conversation. Second we implement a real time user impersonation check with a two step verification process to confirm the callers identity ensuring accountability. A key innovation of our system is the ability to update policies without retraining the entire model enhancing its adaptability. We validated our RAG based approach using synthetic call recordings achieving an accuracy of 97.98 percent and an F1score of 97.44 percent with 100 calls outperforming state of the art methods. This robust and flexible fraud detection system is well suited for real world deployment.

Advanced Real-Time Fraud Detection Using RAG-Based LLMs

TL;DR

This work tackles real-time fraud detection in phone communications by integrating Retrieval-Augmented Generation (RAG) with policy-driven reasoning. It presents a two-pronged system: real-time transcription with a policy-compliance check and a robust two-step user impersonation verification, all capable of updating policies without full model retraining. Evaluated on synthetic call data against BERT and untrained LLM baselines, the RAG approach achieves up to approximately 0.98 accuracy and establishes superior performance while remaining adaptable to evolving policies. The methodology promises practical deployment potential, combining up-to-date policy retrieval, transparent decision-making, and secure data handling to mitigate AI-enabled fraud in real-world scenarios.

Abstract

Artificial Intelligence has become a double edged sword in modern society being both a boon and a bane. While it empowers individuals it also enables malicious actors to perpetrate scams such as fraudulent phone calls and user impersonations. This growing threat necessitates a robust system to protect individuals In this paper we introduce a novel real time fraud detection mechanism using Retrieval Augmented Generation technology to address this challenge on two fronts. First our system incorporates a continuously updating policy checking feature that transcribes phone calls in real time and uses RAG based models to verify that the caller is not soliciting private information thus ensuring transparency and the authenticity of the conversation. Second we implement a real time user impersonation check with a two step verification process to confirm the callers identity ensuring accountability. A key innovation of our system is the ability to update policies without retraining the entire model enhancing its adaptability. We validated our RAG based approach using synthetic call recordings achieving an accuracy of 97.98 percent and an F1score of 97.44 percent with 100 calls outperforming state of the art methods. This robust and flexible fraud detection system is well suited for real world deployment.
Paper Structure (23 sections, 1 equation, 2 figures, 3 tables)

This paper contains 23 sections, 1 equation, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Call Fraud Detection using Call data Records
  4. Call Fraud Detection using Call Contents
  5. Advancements of LLMs in Fraud Detection
  6. Methodology
  7. Dataset Collection
  8. Feature Extraction using LLM
  9. Retrieval-Augmented Generation (RAG)
  10. LLM-Based Policy Compliance Check
  11. Real Time Deployment
  12. Encryption Block
  13. Encryption and Decryption Process
  14. Local Storage and User Consent
  15. Automatic Speech Recognition (ASR)
  16. ...and 8 more sections

Figures (2)

  • Figure 1: Comprehensive Flowchart of the Methodological Framework Employed in the Study
  • Figure 2: Comprehensive Flowchart of the Methodological Framework Employed in the Study