Autonomous LLM-Enhanced Adversarial Attack for Text-to-Motion

TL;DR

ALERT-Motion addresses safety concerns in text-to-motion generation by introducing an autonomous LLM-driven adversarial attack that targets black-box T2M models. It formalizes the attack as maximizing $s^{m}(G(p), m_t)$ over prompts $p\in P$ while keeping $s^{p}(p, p_t) < \eta$ through the adaptive dispatching (AD) and multimodal information contrastive (MMIC) modules, enabling natural and fluent adversarial prompts. Across two T2M models on the HumanML3D dataset, ALERT-Motion achieves higher attack success and more motion-relevant prompts than adapted T2I baselines, illustrating the potency and stealth of the approach. The results underscore urgent needs for defense and responsible deployment in motion-generation systems as these technologies scale.

Abstract

Human motion generation driven by deep generative models has enabled compelling applications, but the ability of text-to-motion (T2M) models to produce realistic motions from text prompts raises security concerns if exploited maliciously. Despite growing interest in T2M, few methods focus on safeguarding these models against adversarial attacks, with existing work on text-to-image models proving insufficient for the unique motion domain. In the paper, we propose ALERT-Motion, an autonomous framework leveraging large language models (LLMs) to craft targeted adversarial attacks against black-box T2M models. Unlike prior methods modifying prompts through predefined rules, ALERT-Motion uses LLMs' knowledge of human motion to autonomously generate subtle yet powerful adversarial text descriptions. It comprises two key modules: an adaptive dispatching module that constructs an LLM-based agent to iteratively refine and search for adversarial prompts; and a multimodal information contrastive module that extracts semantically relevant motion information to guide the agent's search. Through this LLM-driven approach, ALERT-Motion crafts adversarial prompts querying victim models to produce outputs closely matching targeted motions, while avoiding obvious perturbations. Evaluations across popular T2M models demonstrate ALERT-Motion's superiority over previous methods, achieving higher attack success rates with stealthier adversarial prompts. This pioneering work on T2M adversarial attacks highlights the urgency of developing defensive measures as motion generation technology advances, urging further research into safe and responsible deployment.

Figures (4)

• Figure 1: Adversarial prompt against T2M model with RIATIG and our ALERT-Motion. Previous methods like RIATIG only perturb prompts through predefined character or word operations, overlooking the integrity and semantics of the prompts. Our ALERT-Motion doesn't require such predefined operations; instead, by multimodal information contrastive (MMIC) module, the language model autonomously learn and perform these operations, dynamically generating adversarial prompts that meet the attack requirements. Under the same input (target and initial prompt), our method captures more natural and fluent prompts related to motion. When these prompts are used to query the victim T2M model, the resulting motion show a stronger resemblance to the target motion. Darker color indicates later frames in the sequence.
• Figure 2: Overview of the proposed ALERT-Motion. ALERT-Motion operates in a black-box setting with two key modules: multimodal information integration module for consolidating information from text and motion into a unified format, and autonomous AD module that learns and executes adversarial prompt search through progresses of expansion, refinement, and update.
• Figure 3: Examples of adversarial attack results against MDM. The first row of text provides the true annotations for each column of target motions, and the first row of motions corresponds to their respective target motions. The following three rows of text correspond to the adversarial prompts obtained by MacPromp, RIATIG, and our proposed ALERT-Motion. The motion-rendered images below the text depict the motions generated by querying the victim model with the adversarial prompts. Darker color indicates later frames in the sequence.
• Figure 4: Examples of adversarial attack results against MLD. The first row of text provides the true annotations for each column of target motions, and the first row of motions corresponds to their respective target motions. The following three rows of text correspond to the adversarial prompts obtained by MacPromp, RIATIG, and our proposed ALERT-Motion. The motion-rendered images below the text depict the motions generated by querying the victim model with the adversarial prompts. Darker color indicates later frames in the sequence.