LoRA as Oracle

TL;DR

A novel LoRA-based oracle framework that leverages low-rank adaptation modules as a lightweight, model-agnostic probe for both backdoor detection and membership inference, and shows that poisoned and member samples induce distinctive low-rank updates that differ significantly from those generated by clean or non-member data.

Abstract

Backdoored and privacy-leaking deep neural networks pose a serious threat to the deployment of machine learning systems in security-critical settings. Existing defenses for backdoor detection and membership inference typically require access to clean reference models, extensive retraining, or strong assumptions about the attack mechanism. In this work, we introduce a novel LoRA-based oracle framework that leverages low-rank adaptation modules as a lightweight, model-agnostic probe for both backdoor detection and membership inference. Our approach attaches task-specific LoRA adapters to a frozen backbone and analyzes their optimization dynamics and representation shifts when exposed to suspicious samples. We show that poisoned and member samples induce distinctive low-rank updates that differ significantly from those generated by clean or non-member data. These signals can be measured using simple ranking and energy-based statistics, enabling reliable inference without access to the original training data or modification of the deployed model.

Figures (7)

• Figure 1: Overview of the proposed LoRA oracle.
• Figure 2: Overview of the Membership Inference mode.
• Figure 3: Overview of the Backdoor Detection mode.
• Figure 4: Geometric separation of LoRA membership signals across different datasets and models.
• Figure 5: Results on Backdoor target detection across different poisoning rates.