Privacy Impact Assessments in the Wild: A Scoping Review
Leonardo Horn Iwaya, Ala Sarah Alaqra, Marit Hansen, Simone Fischer-Hübner
TL;DR
This scoping review maps PIAs conducted in real-world settings to understand how they are applied, what benefits they deliver, and what barriers impede their effectiveness. By synthesizing 45 studies through a PRISMA-guided protocol, the work highlights a predominantly qualitative, practice-oriented evidence base centered on public-sector contexts and GDPR-driven DPIA considerations. The review identifies widespread use of established PIA frameworks (e.g., ICO, CNIL, OAIC) alongside ad-hoc approaches, while revealing methodological weaknesses, limited empirical validation, and substantial gaps in population diversity. The findings underscore the need for more rigorous, quantitative evaluations and broader cross-cultural studies to move PIAs toward evidence-based privacy engineering and more comprehensive rights-based impact assessments.
Abstract
Privacy Impact Assessments (PIAs) offer a systematic process for assessing the privacy impacts of a project or system. As a privacy engineering strategy, PIAs are heralded as one of the main approaches to privacy by design, supporting the early identification of threats and controls. However, there is still a shortage of empirical evidence on their uptake and proven effectiveness in practice. To better understand the current state of literature and research, this paper provides a comprehensive Scoping Review (ScR) on the topic of PIAs "in the wild", following the well-established Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. As a result, this ScR includes 45 studies, providing an extensive synthesis of the existing body of knowledge, classifying types of research and publications, appraising the methodological quality of primary research, and summarising the positive and negative aspects of PIAs in practice, as reported by studies. This ScR also identifies significant research gaps (e.g., evidence gaps from contradictory results and methodological gaps from research design deficiencies), future research pathways, and implications for researchers, practitioners, and policymakers developing and evaluating PIA frameworks. As we conclude, there is still a significant need for more primary research on the topic, both qualitative and quantitative. A critical appraisal of qualitative studies (n=28) revealed deficiencies in the methodological quality, and only four quantitative studies were identified, suggesting that current primary research remains incipient. Nonetheless, PIAs can be regarded as a prominent sub-area in the broader field of Empirical Privacy Engineering, warranting further research toward more evidence-based practices.