DualTAP: A Dual-Task Adversarial Protector for Mobile MLLM Agents

TL;DR

This work addresses the privacy risk posed by mobile GUI agents where untrusted routers can exploit MLLMs to extract PII from screenshot streams. It introduces DualTAP, a dual-task adversarial protector that uses a contrastive attention module to target privacy-relevant regions and a dual-objective loss to simultaneously minimize privacy leakage and preserve agent utility, with inference deployed on-device. A new PrivScreen benchmark is released to evaluate both privacy leakage and task performance across diverse MLLMs. Experiments on six MLLMs show DualTAP achieves state-of-the-art privacy protection, reducing leakage by about 31.6 percentage points (3.0x) while maintaining roughly 80.8% task success, demonstrating strong practical potential for privacy-preserving mobile AI agents. The approach enables efficient, on-device deployment without large pre-training, offering a concrete path toward mitigating privacy risks in real-world GUI automation tasks.

Abstract

The reliance of mobile GUI agents on Multimodal Large Language Models (MLLMs) introduces a severe privacy vulnerability: screenshots containing Personally Identifiable Information (PII) are often sent to untrusted, third-party routers. These routers can exploit their own MLLMs to mine this data, violating user privacy. Existing privacy perturbations fail the critical dual challenge of this scenario: protecting PII from the router's MLLM while simultaneously preserving task utility for the agent's MLLM. To address this gap, we propose the Dual-Task Adversarial Protector (DualTAP), a novel framework that, for the first time, explicitly decouples these conflicting objectives. DualTAP trains a lightweight generator using two key innovations: (i) a contrastive attention module that precisely identifies and targets only the PII-sensitive regions, and (ii) a dual-task adversarial objective that simultaneously minimizes a task-preservation loss (to maintain agent utility) and a privacy-interference loss (to suppress PII leakage). To facilitate this study, we introduce PrivScreen, a new dataset of annotated mobile screenshots designed specifically for this dual-task evaluation. Comprehensive experiments on six diverse MLLMs (e.g., GPT-5) demonstrate DualTAP's state-of-the-art protection. It reduces the average privacy leakage rate by 31.6 percentage points (a 3.0x relative improvement) while, critically, maintaining an 80.8% task success rate - a negligible drop from the 83.6% unprotected baseline. DualTAP presents the first viable solution to the privacy-utility trade-off in mobile MLLM agents.

Figures (6)

• Figure 1: The Dual Task Completion workflow of the GUI Agent, which is structured upon the Perceive-Router-MLLM paradigm. The figure presents two task flows: (a) In the absence of protection, the task flow achieves its functional objectives but inevitably leaks privacy tasks. (b) Deploying our method on the user side mitigates privacy leakage while preserving normal functionality.
• Figure 2: Overview of the proposed DualTAP framework. (a) Training Phase: A perturbation generator $\mathcal{G}_\phi$ is trained using a dual-task objective. It learns to produce a perturbation $\delta$ based on a contrastive attention map $\mathcal{A}(x)$, guided by a surrogate MLLM $\mathcal{M}_\theta$. The objective is to simultaneously minimize the task-preservation loss $\mathcal{L}_{\text{n}}$ (for normal tasks $\mathcal{Q}_n$) and the privacy-interference loss $\mathcal{L}_{\text{p}}$ (for privacy tasks $\mathcal{Q}_p$). (b) Inference Phase: The pre-trained generator is deployed on the mobile device. It adds the optimized perturbation $\delta$ to the raw screenshot $x$ to create $x_{\mathrm{adv}}$. This $x_{\mathrm{adv}}$ is sent to the untrusted router, which blocks the router's Privacy Extraction MLLM ($\mathcal{M}_e$) from extracting PII, while still allowing the agent's Normal Task MLLM ($\mathcal{M}_a$) to function correctly.
• Figure 3: Comparison of adversarial images generated by different baselines after adding perturbations to the same privacy-sensitive image.
• Figure 4: Comparison of the performance of five different module combinations on the Mobile GUI Agent.
• Figure 5: Impact of the different perturbation bounds