Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Invoice Information Extraction: Methods and Performance Evaluation

Sai Yashwant, Anurag Dubey, Praneeth Paikray, Gantala Thulsiram

TL;DR

The paper addresses the problem of extracting structured invoice data from diverse layouts and languages. It compares a Docling-based OCR/layout approach with an LLM-based LlamaExtractor using a standardized evaluation framework focused on field-level precision, consistency, and exact-match accuracy. Key findings show the LlamaExtractor achieving 94% overall accuracy versus 63% for Docling, with notable gains in robustness and line-item extraction, albeit with higher compute requirements. The work highlights the practical trade-offs between semantic understanding and processing cost, informing deployment decisions for enterprise-grade invoice automation.

Abstract

This paper presents methods for extracting structured information from invoice documents and proposes a set of evaluation metrics (EM) to assess the accuracy of the extracted data against annotated ground truth. The approach involves pre-processing scanned or digital invoices, applying Docling and LlamaCloud Services to identify and extract key fields such as invoice number, date, total amount, and vendor details. To ensure the reliability of the extraction process, we establish a robust evaluation framework comprising field-level precision, consistency check failures, and exact match accuracy. The proposed metrics provide a standardized way to compare different extraction methods and highlight strengths and weaknesses in field-specific performance.

Invoice Information Extraction: Methods and Performance Evaluation

TL;DR

The paper addresses the problem of extracting structured invoice data from diverse layouts and languages. It compares a Docling-based OCR/layout approach with an LLM-based LlamaExtractor using a standardized evaluation framework focused on field-level precision, consistency, and exact-match accuracy. Key findings show the LlamaExtractor achieving 94% overall accuracy versus 63% for Docling, with notable gains in robustness and line-item extraction, albeit with higher compute requirements. The work highlights the practical trade-offs between semantic understanding and processing cost, informing deployment decisions for enterprise-grade invoice automation.

Abstract

This paper presents methods for extracting structured information from invoice documents and proposes a set of evaluation metrics (EM) to assess the accuracy of the extracted data against annotated ground truth. The approach involves pre-processing scanned or digital invoices, applying Docling and LlamaCloud Services to identify and extract key fields such as invoice number, date, total amount, and vendor details. To ensure the reliability of the extraction process, we establish a robust evaluation framework comprising field-level precision, consistency check failures, and exact match accuracy. The proposed metrics provide a standardized way to compare different extraction methods and highlight strengths and weaknesses in field-specific performance.
Paper Structure (23 sections, 2 figures, 4 tables)

This paper contains 23 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Literature Survey
  3. Traditional and Rule Based Methods
  4. Machine Learning and Deep Learning Approaches
  5. Text and Layout-Aware Models
  6. Graph-Based Methods
  7. Large Language Models in Document Information Extraction
  8. Methodology
  9. Dataset and Preprocessing
  10. Method 1: Docling-Based Extraction
  11. Method 2: Llama Extractor
  12. Evaluation Metrics
  13. Results and Analysis
  14. Result - Docling
  15. Result - LlamaExtractor
  16. ...and 8 more sections

Figures (2)

  • Figure 1: Architecture Diagram of the Docling System
  • Figure 2: Architecture Diagram of LlamaExtractor