Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LMV-RPA: Large Model Voting-based Robotic Process Automation

Osama Abdellatif, Ahmed Ayman, Ali Hamdi

TL;DR

LMV-RPA addresses the challenge of automating high-volume unstructured document processing by leveraging a majority voting mechanism across outputs from multiple OCR engines and two large language models to generate structured JSON. The approach combines PaddleOCR, Tesseract, EasyOCR, and DocTR with LLaMA 3 and Gemini-1.5-pro to improve accuracy to 99% and accelerate throughput by about 80% over single-layer configurations. Benchmark results against UiPath and Automation Anywhere demonstrate superior speed and reliability for large-scale OCR workflows, particularly in complex layouts. The work provides a scalable, automatic pipeline for converting unstructured OCR text into structured data, with accessible data and code resources.

Abstract

Automating high-volume unstructured data processing is essential for operational efficiency. Optical Character Recognition (OCR) is critical but often struggles with accuracy and efficiency in complex layouts and ambiguous text. These challenges are especially pronounced in large-scale tasks requiring both speed and precision. This paper introduces LMV-RPA, a Large Model Voting-based Robotic Process Automation system to enhance OCR workflows. LMV-RPA integrates outputs from OCR engines such as Paddle OCR, Tesseract OCR, Easy OCR, and DocTR with Large Language Models (LLMs) like LLaMA 3 and Gemini-1.5-pro. Using a majority voting mechanism, it processes OCR outputs into structured JSON formats, improving accuracy, particularly in complex layouts. The multi-phase pipeline processes text extracted by OCR engines through LLMs, combining results to ensure the most accurate outputs. LMV-RPA achieves 99 percent accuracy in OCR tasks, surpassing baseline models with 94 percent, while reducing processing time by 80 percent. Benchmark evaluations confirm its scalability and demonstrate that LMV-RPA offers a faster, more reliable, and efficient solution for automating large-scale document processing tasks.

LMV-RPA: Large Model Voting-based Robotic Process Automation

TL;DR

LMV-RPA addresses the challenge of automating high-volume unstructured document processing by leveraging a majority voting mechanism across outputs from multiple OCR engines and two large language models to generate structured JSON. The approach combines PaddleOCR, Tesseract, EasyOCR, and DocTR with LLaMA 3 and Gemini-1.5-pro to improve accuracy to 99% and accelerate throughput by about 80% over single-layer configurations. Benchmark results against UiPath and Automation Anywhere demonstrate superior speed and reliability for large-scale OCR workflows, particularly in complex layouts. The work provides a scalable, automatic pipeline for converting unstructured OCR text into structured data, with accessible data and code resources.

Abstract

Automating high-volume unstructured data processing is essential for operational efficiency. Optical Character Recognition (OCR) is critical but often struggles with accuracy and efficiency in complex layouts and ambiguous text. These challenges are especially pronounced in large-scale tasks requiring both speed and precision. This paper introduces LMV-RPA, a Large Model Voting-based Robotic Process Automation system to enhance OCR workflows. LMV-RPA integrates outputs from OCR engines such as Paddle OCR, Tesseract OCR, Easy OCR, and DocTR with Large Language Models (LLMs) like LLaMA 3 and Gemini-1.5-pro. Using a majority voting mechanism, it processes OCR outputs into structured JSON formats, improving accuracy, particularly in complex layouts. The multi-phase pipeline processes text extracted by OCR engines through LLMs, combining results to ensure the most accurate outputs. LMV-RPA achieves 99 percent accuracy in OCR tasks, surpassing baseline models with 94 percent, while reducing processing time by 80 percent. Benchmark evaluations confirm its scalability and demonstrate that LMV-RPA offers a faster, more reliable, and efficient solution for automating large-scale document processing tasks.

Paper Structure

This paper contains 13 sections, 5 equations, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Research Methodology
  4. LMV-RPA Model
  5. Directory Monitoring and File Detection
  6. Multi-OCR Text Extraction
  7. Data Structuring via Large Language Models (LLMs)
  8. Majority Voting for Final JSON
  9. Continuous Automation
  10. Benchmark and Experimental Design
  11. Results and Discussion
  12. Conclusion
  13. Data and Code Availability

Figures (1)

  • Figure 1: System architecture of the proposed LMV-RPA framework. The process starts with monitoring any particular folder for an image upload (Process 1). Once an image is detected, it is processed by four different OCR engines such as Paddle OCR, Tesseract OCR, Easy OCR, and DocTR OCR in Process 2, Process 3, Process 4, and Process 5, respectively. The outputs from these OCR engines go through two different Large Language Models, LLM1 and LLM2, which convert the extracted text into JSON format. in Processes from 6 to 9 handling the text from Paddle, EasyOCR, Tesseract, and DocTR by LLM1, while the same is done by LLM2 in Processes from 10 through 13. This output undergoes a majority voting mechanism in (Process 14) for selecting the most accurate JSON output to ensure reliable and precise text extraction from complex document layouts. The final structured data is in JSON format.