sudoLLM: On Multi-role Alignment of Language Models

TL;DR

sudoLLM introduces a user-aware, multi-role alignment framework that injects an authorization-based bias into user queries and fine-tunes an LLM to respond or refuse based on user privileges. By leveraging biased query rephrasing and targeted fine-tuning, the approach improves safety alignment, generalizes to out-of-distribution data, and enhances resistance to prefix-based jailbreaks while maintaining performance on allowed content. The method operates as a complementary security layer to existing guardrails and can be deployed in black-box API settings with standard authentication. Empirical results across medical and legal domains demonstrate stronger Bob refusals, fails-closed behavior, and detectable adversarial tampering, with modest data and computation requirements. This work offers a practical path to enforce role-based access in LLMs for regulated applications and parental controls, while highlighting important security and ethical considerations.

Abstract

User authorization-based access privileges are a key feature in many safety-critical systems, but have not been extensively studied in the large language model (LLM) realm. In this work, drawing inspiration from such access control systems, we introduce sudoLLM, a novel framework that results in multi-role aligned LLMs, i.e., LLMs that account for, and behave in accordance with, user access rights. sudoLLM injects subtle user-based biases into queries and trains an LLM to utilize this bias signal in order to produce sensitive information if and only if the user is authorized. We present empirical results demonstrating that this approach shows substantially improved alignment, generalization, resistance to prefix-based jailbreaking attacks, and ``fails-closed''. The persistent tension between the language modeling objective and safety alignment, which is often exploited to jailbreak LLMs, is somewhat resolved with the aid of the injected bias signal. Our framework is meant as an additional security layer, and complements existing guardrail mechanisms for enhanced end-to-end safety with LLMs.

Figures (6)

• Figure 1: Envisioned multi-role alignment with sudoLLM paradigm.Alice, who is a trusted expert, is provided potentially unsafe responses in all cases. Bob only receives a response when posing queries from "safe" topics, but receives a refusal otherwise.
• Figure 2: Schematic diagram of sudoLLM paradigm. During training (left) two versions of a query are generated (see Eq. \ref{['eq:modDist']}), which are then used to fine-tune an LLM to answer queries when coming from Alice or provide a refusal when coming from Bob. (Right) shows the inference procedure.
• Figure 3: Examples of OQ, RQ and BQ.
• Figure 4: Attack strategy. Adding a suffix to the user query was found to not work, however, augmenting the query with a user request for continuation was successful.
• Figure 5: Splits of LegalBench used in the study.