Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agentic Framework for Political Biography Extraction

Yifei Zhu, Songpo Yang, Jiangnan Zhu, Junyan Jiang

Abstract

The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.

Agentic Framework for Political Biography Extraction

Abstract

The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.
Paper Structure (85 sections, 14 equations, 14 figures, 14 tables)

This paper contains 85 sections, 14 equations, 14 figures, 14 tables.

Table of Contents

  1. Introduction
  2. The Challenge of Extracting Political Biographies at Scale
  3. From Classification to Extraction: The Context Challenge
  4. An Agentic Solution to the Extraction Challenge
  5. Experiment 1: The Coding Challenge
  6. Data
  7. Human-Coded Benchmark: The CPED
  8. Sample Construction
  9. Evaluation Design
  10. CGT Construction
  11. Performance Metrics
  12. Estimation Strategy
  13. Results
  14. Experiment 2: The Synthesis Challenge
  15. Data
  16. ...and 70 more sections

Figures (14)

  • Figure 1: Two coding strategies for elite biographies. Left: when a Wikipedia page exists, we code directly from the curated page with a single LLM pass. Right: when Wikipedia is missing or incomplete, we search across web sources and iteratively synthesize a synthetic report, then code from that report. The lower panel illustrates the structured output as an ordered biography (career, education, and affiliations) anchored on a timeline. This contrast highlights why extraction from open-web sources requires adaptive synthesis rather than one-shot retrieval.
  • Figure 2: Experiment 1 Results: LLM coding performance relative to the human baseline (China sample, N=197). Points indicate coefficient estimates with 95% confidence intervals. The human-coded baseline (Human_wiki) is normalized to zero. Positive values indicate that LLMs outperform human coders on the corresponding metric.
  • Figure 3: Agentic synthesis versus Wikipedia baseline (pooled U.S. and OECD samples, $N=398$). Points indicate coefficient estimates from Equation \ref{['eq:rq2_spec']} with 95% confidence intervals. The Wikipedia baseline (LLM_wiki) is normalized to zero. Positive values indicate that agentic synthesis outperforms Wikipedia-based extraction.
  • Figure 4: Agentic synthesis effects by sample. Points report F1 estimates with 95% confidence intervals. Baselines are normalized to sample-specific Wikipedia means (U.S.: 0.82; OECD: 0.73).
  • Figure 5: Refined versus raw corpora: the refinement premium.
  • ...and 9 more figures