FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information

TL;DR

FinTagging delivers the first structure-aware, full-scope XBRL tagging benchmark designed for LLM evaluation in zero-shot settings. It splits tagging into FinNI for numeric extraction and FinCL for concept linking, and evaluates models on text and tables derived from real 10-K filings. Results show LLMs excel at numeric identification but struggle with precise GAAP concept alignment across a large taxonomy, underscoring the need for better semantic grounding and retrieval-augmented or hybrid approaches for accurate financial disclosure tagging. The benchmark provides a rigorous, scalable framework and data for advancing end-to-end financial tagging and regulatory reporting capabilities.

Abstract

Accurately understanding numbers from financial reports is fundamental to how markets, regulators, algorithms, and normal people read the economy and the world, yet even with XBRL (eXtensible Business Reporting Language) designed to tag every figure with standardized accounting concepts, mapping thousands of facts to over 10,000 U.S. GAAP concepts remains costly, inconsistent, and error-prone. Existing benchmarks define tagging as flat, single-step, extreme classification over small subsets of US-GAAP concepts, overlooking both the taxonomy's hierarchical semantics and the structured nature of real tagging, where each fact must be represented as a contextualized multi-field output. These simplifications prevent fair evaluation of large language models (LLMs) under realistic reporting conditions. To address these gaps, we introduce FinTagging, the first comprehensive benchmark for structure-aware and full-scope XBRL tagging, designed to evaluate LLMs' ability to extract and align financial facts through numerical reasoning and taxonomy alignment across text and tables. We define two subtasks: FinNI for numeric identification, which extracts numerical entities and their types from XBRL reports, and FinCL for concept linking, which maps each extracted entity to the corresponding concept in the full US-GAAP taxonomy. Together, these subtasks produce a structured representation of each financial fact. We evaluate diverse LLMs under zero-shot settings and analyze their performance across both subtasks and overall tagging accuracy. Results show that LLMs generalize well in numeric identification but struggle with fine-grained concept linking, revealing current limitations in structure-aware reasoning for accurate financial disclosure. All code and datasets are available on GitHub and Hugging Face.

Figures (11)

• Figure 1: An example of Financial Tagging in Realistic, where the orange numbers mark facts to be tagged, black dashed arrows show the structured outputs after tagging, and blue dashed arrows denote the US-GAAP taxonomy referenced during tagging.
• Figure 2: Statistics of numerical entity types.
• Figure 3: A unified evaluation framework on FinTagging57,126,255240,89,215 benchmark. Note: The Sentence is drawn from financial report $D$, and the Query provides an example that incorporates the entity 4.9. The black arrows show the overall framework flow, green arrows denote the FinNI process, red arrows indicate the FinCL process, red bidirectional arrows represent retrieval interactions with the taxonomy, and red dashed arrows mark the recomposed input between the query and retrieved candidates.
• Figure 4: Visualization of significance analysis across models. (a) shows the pairwise-bootstrap significance matrix, while (b) summarizes how many significant differences each model has relative to others.
• Figure 5: Distribution of Tickers by Industry Sector.