Large-scale data extraction from the UNOS organ donor documents

TL;DR

The paper tackles the challenge of making OPTN donor data stored in PDF attachments accessible for large-scale analysis. It develops a pipeline that converts native PDFs from DonorNet into structured data using text extraction with PDFBox, image processing to decode checkboxes, and regular grammar/regex-based parsing. It demonstrates case studies on DCD flowsheets and liver data, builds a scalable 18-table database and supports exports to SQL, MongoDB, and JSON, and shows promising performance on HIPAA-compliant HPC. The work enables applying the method to the full OPTN dataset and future data, offering a path toward more comprehensive analyses that could improve organ utilization and patient outcomes.

Abstract

In this paper we focus on three major task: 1) discussing our methods: Our method captures a portion of the data in DCD flowsheets, kidney perfusion data, and Flowsheet data captured peri-organ recovery surgery. 2) demonstrating the result: We built a comprehensive, analyzable database from 2022 OPTN data. This dataset is by far larger than any previously available even in this preliminary phase; and 3) proving that our methods can be extended to all the past OPTN data and future data. The scope of our study is all Organ Procurement and Transplantation Network (OPTN) data of the USA organ donors since 2008. The data was not analyzable in a large scale in the past because it was captured in PDF documents known as ``Attachments'', whereby every donor's information was recorded into dozens of PDF documents in heterogeneous formats. To make the data analyzable, one needs to convert the content inside these PDFs to an analyzable data format, such as a standard SQL database. In this paper we will focus on 2022 OPTN data, which consists of $\approx 400,000$ PDF documents spanning millions of pages. The entire OPTN data covers 15 years (2008--20022). This paper assumes that readers are familiar with the content of the OPTN data.

Figures (3)

• Figure 1: A DCD FLowsheet Variant (PII redacted).
• Figure 2: DCD Flowsheet --- graphical objects and binarization.
• Figure 3: The LIVER DATA form --- graphical objects and binarization (PII redacted).