AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents

Abstract

The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.

Figures (7)

• Figure 1: Overview of our backdoor framework. The framework leverages a compromised MLLM to execute adversary-intended actions when triggered by benign-looking notifications. By leveraging contact details exposed in recent data breaches (e.g., the Optus incident optus_data_breach), adversaries can easily target specific users to deliver these notification triggers.
• Figure 2: Overview of AgentRAE: First, we build poisoned data along task navigation in two phases. Then, we apply two-phase training: Phase 1 separates trigger representations, while Phase 2 injects multi-target backdoors with utility preserved. Finally, the notification trigger activates the adversary-intended action from the action space supported by existing mobile GUI agents.
• Figure 3: Examples of poisoned sample construction in two phases. Phase 1: Only poisoned samples with different notification triggers for contrastive learning. Phase 2: Mix clean, poisoned, and benign icon samples for backdoor fine-tuning.
• Figure 4: t-SNE visualization of trigger representations before and after Phase 1 contrastive learning.
• Figure 5: Backdoor robustness on OdysseyAgent-app: poison size and resolution.