Toward Regulatory Compliance: A few-shot Learning Approach to Extract Processing Activities
Pragyan KC, Rambod Ghandiparsi, Rocky Slavin, Sepideh Ghanavati, Travis Breaux, Mitra Bokaei Hosseini
TL;DR
The paper tackles GDPR RoPA compliance for small app developers by enabling automatic RoPA segment generation from user usage scenarios using few-shot learning with GPT-3.5 Turbo. It introduces a three-task pipeline with scenario collection, concept identification, and controlled NL templates for annotation and summarization, and provides a 50-scenario corpus with summarized processing activities. The study systematically analyzes prompt design factors, showing that increasing the number of in-prompt examples substantially improves ROUGE-L performance while prompt repetition and example order have minimal impact, achieving an average ROUGE-L F1 of about $0.70$ on testing data. The work demonstrates practical potential for SME privacy compliance tools, while highlighting the need to compare other LLMs and incorporate NER and manual evaluation in future work.
Abstract
The widespread use of mobile applications has driven the growth of the industry, with companies relying heavily on user data for services like targeted advertising and personalized offerings. In this context, privacy regulations such as the General Data Protection Regulation (GDPR) play a crucial role. One of the GDPR requirements is the maintenance of a Record of Processing Activities (RoPA) by companies. RoPA encompasses various details, including the description of data processing activities, their purposes, types of data involved, and other relevant external entities. Small app-developing companies face challenges in meeting such compliance requirements due to resource limitations and tight timelines. To aid these developers and prevent fines, we propose a method to generate segments of RoPA from user-authored usage scenarios using large language models (LLMs). Our method employs few-shot learning with GPT-3.5 Turbo to summarize usage scenarios and generate RoPA segments. We evaluate different factors that can affect few-shot learning performance consistency for our summarization task, including the number of examples in few-shot learning prompts, repetition, and order permutation of examples in the prompts. Our findings highlight the significant influence of the number of examples in prompts on summarization F1 scores, while demonstrating negligible variability in F1 scores across multiple prompt repetitions. Our prompts achieve successful summarization of processing activities with an average 70% ROUGE-L F1 score. Finally, we discuss avenues for improving results through manual evaluation of the generated summaries.