PRITES: An integrative framework for investigating and assessing web-scraped HTTP-response datasets for research applications
Cynthia A. Huang, Tina Lam
TL;DR
Web-scraped datasets are widespread but pose challenges for statistical validity due to non-representativeness and opaque data collection processes. The authors propose the PRITES framework to structure planning, retrieval, investigation, transformation, evaluation, and summarisation of web-response datasets, integrating technical, provenance, and statistical considerations. The framework supports documentation, reproducibility, and multidisciplinary collaboration, and is demonstrated via a case study adapting a commercially collected web-scraped alcohol price dataset for public health research. This work offers a practical, generalizable workflow and provenance artefacts to improve data quality assessment and reporting for web-response data, with broad implications for researchers, data providers, and policy applications.
Abstract
The ability to programmatically retrieve vast quantities of data from online sources has given rise to increasing usage of web-scraped datasets for various purposes across government, industry and academia. Contemporaneously, there has also been growing discussion about the statistical qualities and limitations of collecting from online data sources and analysing web-scraped datasets. However, literature on web-scraping is distributed across computer science, statistical methodology and application domains, with distinct and occasionally conflicting definitions of web-scraping and conceptualisations of web-scraped data quality. This work synthesises technical and statistical concepts, best practices and insights across these relevant disciplines to inform documentation during web-scraping processes, and quality assessment of the resultant web-scraped datasets. We propose an integrated framework to cover multiple processes during the creation of web-scraped datasets including 'Plan', 'Retrieve', 'Investigate', 'Transform', 'Evaluate' and 'Summarise' (PRITES). The framework groups related quality factors which should be monitored during the collection of new web-scraped data, and/or investigated when assessing potential applications of existing web-scraped datasets. We connect each stage to existing discussions of technical and statistical challenges in collecting and analysing web-scraped data. We then apply the framework to describe related work by the co-authors to adapt web-scraped retail prices for alcoholic beverages collected by an industry data partner into analysis-ready datasets for public health policy research. The case study illustrates how the framework supports accurate and comprehensive scientific reporting of studies using web-scraped datasets.