TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data

TL;DR

TrustLoRA addresses safe, open-world prediction by separating reliability knowledge into low-rank adapters and unifying them through LoRA arithmetic. By freezing the backbone and learning covariate-shift reliability with AugMix ($\mathcal{L}_{\rm LoRA,cov}$) and semantic-shift reliability with OE ($\mathcal{L}_{\rm LoRA,sem}$), two LoRA modules capture distinct failure sources, then merge via $\tau = \sum_t \alpha_t \tau_{\rm LoRA,t}$ to form a single, controllable detector. The approach supports rank adaptation through random projection and enables flexible reliability editing (addition or negation) to balance misclassification rejection and semantic OOD rejection without full retraining. Across CIFAR-10/100-C and ImageNet experiments, TrustLoRA outperforms strong baselines on unified failure detection metrics (e.g., AURC) while maintaining ID accuracy, and it demonstrates robustness to auxiliary data choices and backbones such as ViT. Overall, the method offers a practical, scalable way to implement flexible reliability under covariate and semantic shifts in open environments.

Abstract

Reliable prediction is an essential requirement for deep neural models that are deployed in open environments, where both covariate and semantic out-of-distribution (OOD) data arise naturally. In practice, to make safe decisions, a reliable model should accept correctly recognized inputs while rejecting both those misclassified covariate-shifted and semantic-shifted examples. Besides, considering the potential existing trade-off between rejecting different failure cases, more convenient, controllable, and flexible failure detection approaches are needed. To meet the above requirements, we propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts. Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters and then integrating them, we can enhance the failure detection ability effectively and flexibly. Extensive experiments demonstrate the superiority of our framework.

Figures (8)

• Figure 1: (a) Failure detection rejects both the (✗) misclassified covariate-shifted and all semantic-shifted OOD samples, and accepts the (✓) correct prediction. (b) Illustration of three types of common failure cases in the natural open environment.
• Figure 2: Covariate shifts complicate failure detection.
• Figure 3: Illustration of the proposed reliability arithmetic framework. (Left) We freeze the pre-trained backbone and add a LoRA module to acquire failure-specific knowledge. (Right) The LoRAs will be merged via arithmetic for unified failure detection in the wild.
• Figure 4: Change of rejection ability when fine-tuning the pre-trained ResNet-18 on CIFAR-100.
• Figure 5: Flexibility of controlling the strength of reliability edition on CIFAR-100.