Legally Binding but Unfair? Towards Assessing Fairness of Privacy Policies
Vincent Freiberger, Erik Buchmann
TL;DR
This work tackles the fairness of privacy policies by formalizing three interrelated dimensions—informational fairness, representational fairness, and ethics/morality—grounded in legal frameworks and social science. It proposes an automated assessment pipeline combining readability and bias metrics with large-language-model–based ethics evaluations to quantify each dimension. Preliminary experiments on 618 German privacy policies from the Top-100 web shops reveal issues across all three dimensions, highlighting risks of unfair communication and representation. The approach aims to increase transparency for data subjects and to support legal NLP models by providing a scalable fairness assessment of privacy policies.
Abstract
Privacy policies are expected to inform data subjects about their data protection rights and should explain the data controller's data management practices. Privacy policies only fulfill their purpose, if they are correctly interpreted, understood, and trusted by the data subject. This implies that a privacy policy is written in a fair way, e.g., it does not use polarizing terms, does not require a certain education, or does not assume a particular social background. We outline our approach to assessing fairness in privacy policies. We identify from fundamental legal sources and fairness research, how the dimensions informational fairness, representational fairness and ethics / morality are related to privacy policies. We propose options to automatically assess policies in these fairness dimensions, based on text statistics, linguistic methods and artificial intelligence. We conduct initial experiments with German privacy policies to provide evidence that our approach is applicable. Our experiments indicate that there are issues in all three dimensions of fairness. This is important, as future privacy policies may be used in a corpus for legal artificial intelligence models.