Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Sensitivity-Aware Language Models

Dren Fazlija, Iyiola E. Olatunji, Daniel Kudenko, Sandipan Sikdar

TL;DR

The paper addresses the risk of sensitive-data leakage by LLMs in corporate settings and formalizes sensitivity awareness (SA) as policy-aware information flow. It builds a formal bridge between SA, attribute inference (AI), and differential privacy (DP) through privacy games and an RBAC-based model, deriving both lower and DP-based upper bounds on SA leakage. Practically, it proposes a resource-efficient supervised fine-tuning pipeline using LoRA on 4-bit quantized Qwen3 models, trained with the Access Denied Inc (ADI) annotations. Empirically, SA can be substantially improved (up to 21.7 percentage points) with modest impacts on general reasoning tasks, enabling on-device deployment with strong privacy guarantees and competitive performance among open-source and commercial models.

Abstract

With LLMs increasingly deployed in corporate data management, it is crucial to ensure that these models do not leak sensitive information. In the context of corporate data management, the concept of sensitivity awareness has been introduced, enabling LLMs to adhere to predefined access rights rules. However, it remains unclear how sensitivity awareness relates to established notions of privacy, such as differential privacy (DP), thereby making it difficult to deploy meaningfully in real-world applications. In this work, we formalize the notion of sensitivity awareness and theoretically establish its connection to DP. Additionally, we develop a supervised fine-tuning recipe to make existing, four-bit quantized LLMs more sensitivity-aware. With a performance boost of up to 21.7%, the finetuned LLMs not only substantially improve over their baseline but also outperform other full-precision open-source and commercial models of similar size in achieving sensitivity awareness, demonstrating the effectiveness of our proposed approach. At the same time, our method also largely preserves the models' performance on other tasks, such as general instruction-following, mathematical, and common-sense reasoning.

Towards Sensitivity-Aware Language Models

TL;DR

The paper addresses the risk of sensitive-data leakage by LLMs in corporate settings and formalizes sensitivity awareness (SA) as policy-aware information flow. It builds a formal bridge between SA, attribute inference (AI), and differential privacy (DP) through privacy games and an RBAC-based model, deriving both lower and DP-based upper bounds on SA leakage. Practically, it proposes a resource-efficient supervised fine-tuning pipeline using LoRA on 4-bit quantized Qwen3 models, trained with the Access Denied Inc (ADI) annotations. Empirically, SA can be substantially improved (up to 21.7 percentage points) with modest impacts on general reasoning tasks, enabling on-device deployment with strong privacy guarantees and competitive performance among open-source and commercial models.

Abstract

With LLMs increasingly deployed in corporate data management, it is crucial to ensure that these models do not leak sensitive information. In the context of corporate data management, the concept of sensitivity awareness has been introduced, enabling LLMs to adhere to predefined access rights rules. However, it remains unclear how sensitivity awareness relates to established notions of privacy, such as differential privacy (DP), thereby making it difficult to deploy meaningfully in real-world applications. In this work, we formalize the notion of sensitivity awareness and theoretically establish its connection to DP. Additionally, we develop a supervised fine-tuning recipe to make existing, four-bit quantized LLMs more sensitivity-aware. With a performance boost of up to 21.7%, the finetuned LLMs not only substantially improve over their baseline but also outperform other full-precision open-source and commercial models of similar size in achieving sensitivity awareness, demonstrating the effectiveness of our proposed approach. At the same time, our method also largely preserves the models' performance on other tasks, such as general instruction-following, mathematical, and common-sense reasoning.
Paper Structure (17 sections, 3 theorems, 19 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 17 sections, 3 theorems, 19 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Formal Foundations for SA
  4. Implication of the General Lower Bound.
  5. Bridging SA, Attribute Inference (AI), and Differential Privacy (DP)
  6. SA and AI: A Behavioral Perspective
  7. Proof of Lemma \ref{['lem:aisa']} (SA $\preceq$ AI).
  8. SA and DP
  9. Proof of Theorem \ref{['cor:advupperbound']}.
  10. Experimental Setup
  11. Models and Training Configuration
  12. Evaluation Framework
  13. General Model Capability Evaluation
  14. Results
  15. Is LoRA Fine-tuning All You Need?
  16. ...and 2 more sections

Key Result

Lemma 1

For any adversary $\mathcal{A}$, the advantage in the sensitivity awareness (SA) game is at most the advantage in the attribute inference (AI) game. Consequently, $SA \preceq AI$.

Figures (3)

  • Figure 1: Visual Overview of Contributions. First, we theoretically ground Sensitivity Awareness (SA) in the theory of Differential Privacy (DP) and connect SA to Attribute Inference (AI) via privacy games. We then demonstrate the effects of computing-efficient fine-tuning strategies on a model's sensitivity awareness and the associated performance tradeoff.
  • Figure 2: Example Outputs. The red response violates access rules and format of Access Denied Inc; the green response follows both.
  • Figure 3: Overall correctness rate across all 10,500 questions. The figure also includes the correctness rate of the best-performing model, Grok-2, of the original ADI study fazlija2025access.

Theorems & Definitions (5)

  • Lemma 1: $SA \preceq AI$
  • Definition 3.1: SA Advantage
  • Theorem 2: General Lower Bound on SA Advantage
  • proof
  • Theorem 3: Upper Bound on SA Advantage via DP