Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

TL;DR

Aegis introduces a structured red-teaming framework to evaluate the governance, integrity, and security of Audio Large Language Model–powered voice agents in high-stakes domains. It models end-to-end deployments across banking, IT support, and logistics, with five adversarial scenarios derived from MITRE ATT&CK and an automated GPT-4o–based attack agent operating under diverse personas. The framework reveals that limiting data access via query-based interfaces mitigates identity and data-exfiltration risks, but behavioral vulnerabilities such as resource abuse and privilege escalation persist, especially for open-weight models. The findings advocate a layered defense approach—combining access control, policy enforcement, and continuous behavioral monitoring—and underscore governance, auditing, and regulatory considerations for safer deployment of next-generation voice agents.

Abstract

With the rapid advancement and adoption of Audio Large Language Models (ALLMs), voice agents are now being deployed in high-stakes domains such as banking, customer service, and IT support. However, their vulnerabilities to adversarial misuse still remain unexplored. While prior work has examined aspects of trustworthiness in ALLMs, such as harmful content generation and hallucination, systematic security evaluations of voice agents are still lacking. To address this gap, we propose Aegis, a red-teaming framework for the governance, integrity, and security of voice agents. Aegis models the realistic deployment pipeline of voice agents and designs structured adversarial scenarios of critical risks, including privacy leakage, privilege escalation, resource abuse, etc. We evaluate the framework through case studies in banking call centers, IT Support, and logistics. Our evaluation shows that while access controls mitigate data-level risks, voice agents remain vulnerable to behavioral attacks that cannot be addressed through access restrictions alone, even under strict access controls. We observe systematic differences across model families, with open-weight models exhibiting higher susceptibility, underscoring the need for layered defenses that combine access control, policy enforcement, and behavioral monitoring to secure next-generation voice agents.

Figures (3)

• Figure 1: Overview of the red-teaming framework for voice agents. The framework evaluates deployed agents in three real-world settings: banking, IT support, and logistics across five adversarial scenarios: authentication bypass, privacy leakage, resource abuse, privilege escalation, and data poisoning. Each scenario simulates realistic attack interactions to assess model behavior, security risks, and policy compliance under adversarial conditions.
• Figure 2: The attack agent is guided by a high-level objective and one of five predefined personas, then engages in a multi-turn dialogue with the target agent. Each response is conditioned on the full conversation history to adapt its strategy. After each attempt, the interaction transcript is evaluated to determine whether the attack succeeded.
• Figure 3: Average difference in attack success rates across all models for different personas and adversarial scenarios, relative to the original results in Table 1. The differences are consistently small (within a few percentage points), indicating that persona choice has minimal impact on the overall attack outcomes.