Privacy-Aware Document Visual Question Answering

TL;DR

The paper tackles privacy concerns in DocVQA by introducing PFL-DocVQA, a large-scale, privacy-focused invoice dataset for federated learning and differential privacy experiments. It demonstrates memorization and provider-leakage risks in state-of-the-art multimodal DocVQA models and proposes attacks (PMIA, AZK, APK) to quantify leakage. It then evaluates privacy-preserving baselines, showing that Federated Learning and Differential Privacy can substantially mitigate leakage, with measurable trade-offs in task utility. The work offers a practical framework and dataset for evaluating privacy in multi-modal document understanding models, with implications for deploying DocVQA in real-world, privacy-sensitive settings.

Abstract

Document Visual Question Answering (DocVQA) has quickly grown into a central task of document understanding. But despite the fact that documents contain sensitive or copyrighted information, none of the current DocVQA methods offers strong privacy guarantees. In this work, we explore privacy in the domain of DocVQA for the first time, highlighting privacy issues in state of the art multi-modal LLM models used for DocVQA, and explore possible solutions. Specifically, we focus on invoice processing as a realistic document understanding scenario, and propose a large scale DocVQA dataset comprising invoice documents and associated questions and answers. We employ a federated learning scheme, that reflects the real-life distribution of documents in different businesses, and we explore the use case where the data of the invoice provider is the sensitive information to be protected. We demonstrate that non-private models tend to memorise, a behaviour that can lead to exposing private information. We then evaluate baseline training schemes employing federated learning and differential privacy in this multi-modal scenario, where the sensitive information might be exposed through either or both of the two input modalities: vision (document image) or language (OCR tokens). Finally, we design attacks exploiting the memorisation effect of the model, and demonstrate their effectiveness in probing a representative DocVQA models.

Figures (12)

• Figure 1: The risk of malicious attacks on trained DocVQA models, such as exploiting memorization, is evident in the PFL-DocVQA dataset. Adversaries can cue the model through the visual modality, to invoke memory and reveal sensitive information that is not explicitly in the document (e.g. in this example the provider's name). We show how this behaviour can be exploited to attack the models, and take first steps to mitigate the problem.
• Figure 2: Distribution of providers and documents across different groups and splits. Every bar represents a specific provider, which contains a set of documents. The BLUE dataset is used for training the models, while the RED data is used for the attacks.
• Figure 3: Question answering performance of the base method VT5. The DP models are trained with $\delta=10^{-5}$.
• Figure 4: The left panel displays attack performance as a function of privacy budget for $s=5$ whereas the right panel displays the performance as a function of $s$ for the non-private model. AZK/APK denotes our zero-knowledge/partial-knowledge setting. Results are reported with the standard deviation over 5 random seeds.
• Figure A.1: Examples of different invoice document images of from PFL-DocVQA.