Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents

Hossein Sholehrasa, Amirhossein Ghanaatian, Doina Caragea, Lisa A. Tell, Jim E. Riviere, Majid Jaberi-Douraki

TL;DR

AutoPK tackles the challenge of extracting pharmacokinetic data from heterogeneous scientific tables by introducing a two-stage framework that first identifies PK parameter variants with a hybrid similarity metric and LLM validation, then reconstructs standardized, CSV-format tables via a table-simplification and LLM-based reconstruction step. The approach combines a preprocessing step, a Variant Identification pipeline, and a Final Table Reconstruction pipeline to address structural variability and terminological inconsistency in PK tables. On a real-world dataset of 605 PK tables, AutoPK, especially with open-source models like Gemma 3-27B or LLaMA 3-70B, achieves high F1-scores across key PK parameters and significantly reduces hallucinations compared with direct LLM usage, often outperforming commercial models in several tasks. The work demonstrates practical, scalable PK data extraction with strong generalization across table formats, enabling more efficient veterinary pharmacology, drug safety monitoring, and public health decision-making.

Abstract

Pharmacokinetics (PK) plays a critical role in drug development and regulatory decision-making for human and veterinary medicine, directly affecting public health through drug safety and efficacy assessments. However, PK data are often embedded in complex, heterogeneous tables with variable structures and inconsistent terminologies, posing significant challenges for automated PK data retrieval and standardization. AutoPK, a novel two-stage framework for accurate and scalable extraction of PK data from complex scientific tables. In the first stage, AutoPK identifies and extracts PK parameter variants using large language models (LLMs), a hybrid similarity metric, and LLM-based validation. The second stage filters relevant rows, converts the table into a key-value text format, and uses an LLM to reconstruct a standardized table. Evaluated on a real-world dataset of 605 PK tables, including captions and footnotes, AutoPK shows significant improvements in precision and recall over direct LLM baselines. For instance, AutoPK with LLaMA 3.1-70B achieved an F1-score of 0.92 on half-life and 0.91 on clearance parameters, outperforming direct use of LLaMA 3.1-70B by margins of 0.10 and 0.21, respectively. Smaller models such as Gemma 3-27B and Phi 3-12B with AutoPK achieved 2-7 fold F1 gains over their direct use, with Gemma's hallucination rates reduced from 60-95% down to 8-14%. Notably, AutoPK enabled open-source models like Gemma 3-27B to outperform commercial systems such as GPT-4o Mini on several PK parameters. AutoPK enables scalable and high-confidence PK data extraction, making it well-suited for critical applications in veterinary pharmacology, drug safety monitoring, and public health decision-making, while addressing heterogeneous table structures and terminology and demonstrating generalizability across key PK parameters. Code and data: https://github.com/hosseinsholehrasa/AutoPK

AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents

TL;DR

AutoPK tackles the challenge of extracting pharmacokinetic data from heterogeneous scientific tables by introducing a two-stage framework that first identifies PK parameter variants with a hybrid similarity metric and LLM validation, then reconstructs standardized, CSV-format tables via a table-simplification and LLM-based reconstruction step. The approach combines a preprocessing step, a Variant Identification pipeline, and a Final Table Reconstruction pipeline to address structural variability and terminological inconsistency in PK tables. On a real-world dataset of 605 PK tables, AutoPK, especially with open-source models like Gemma 3-27B or LLaMA 3-70B, achieves high F1-scores across key PK parameters and significantly reduces hallucinations compared with direct LLM usage, often outperforming commercial models in several tasks. The work demonstrates practical, scalable PK data extraction with strong generalization across table formats, enabling more efficient veterinary pharmacology, drug safety monitoring, and public health decision-making.

Abstract

Pharmacokinetics (PK) plays a critical role in drug development and regulatory decision-making for human and veterinary medicine, directly affecting public health through drug safety and efficacy assessments. However, PK data are often embedded in complex, heterogeneous tables with variable structures and inconsistent terminologies, posing significant challenges for automated PK data retrieval and standardization. AutoPK, a novel two-stage framework for accurate and scalable extraction of PK data from complex scientific tables. In the first stage, AutoPK identifies and extracts PK parameter variants using large language models (LLMs), a hybrid similarity metric, and LLM-based validation. The second stage filters relevant rows, converts the table into a key-value text format, and uses an LLM to reconstruct a standardized table. Evaluated on a real-world dataset of 605 PK tables, including captions and footnotes, AutoPK shows significant improvements in precision and recall over direct LLM baselines. For instance, AutoPK with LLaMA 3.1-70B achieved an F1-score of 0.92 on half-life and 0.91 on clearance parameters, outperforming direct use of LLaMA 3.1-70B by margins of 0.10 and 0.21, respectively. Smaller models such as Gemma 3-27B and Phi 3-12B with AutoPK achieved 2-7 fold F1 gains over their direct use, with Gemma's hallucination rates reduced from 60-95% down to 8-14%. Notably, AutoPK enabled open-source models like Gemma 3-27B to outperform commercial systems such as GPT-4o Mini on several PK parameters. AutoPK enables scalable and high-confidence PK data extraction, making it well-suited for critical applications in veterinary pharmacology, drug safety monitoring, and public health decision-making, while addressing heterogeneous table structures and terminology and demonstrating generalizability across key PK parameters. Code and data: https://github.com/hosseinsholehrasa/AutoPK

Paper Structure

This paper contains 22 sections, 3 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Definition and Motivation
  3. AutoPK
  4. Preprocessing
  5. Pipeline 1: PK Parameter Variant Identification
  6. Initial Extraction of Candidate PK Parameter Variants
  7. Table-Wide Matching Using Hybrid Similarity Scoring
  8. LLM-Based Validation of New Variants
  9. Pipeline 2: Final Table Reconstruction
  10. Table Simplification
  11. Table-to-Text Conversion and LLM-based Table Reconstruction
  12. Baseline: LLM Direct Table Retrieval
  13. Experimental Setup
  14. Dataset
  15. Experimental Settings
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Overview of the AutoPK table retrieval process. PK tables are preprocessed and scanned to extract initial AUC-related parameter variants. Expand the variants list via exact and hybrid similarity matching. Validated variants from an LLM are filtered and used to reconstruct structured CSV output.
  • Figure 2: The prompts used in AutoPK are summarized for (A) extracting PK parameter variants and (B) reconstructing CSV tables from simplified key-value text representations. Variables like {pk parameter}, {variants Aliases}, and table metadata are dynamically replaced at runtime.