Center-Based Relaxed Learning Against Membership Inference Attacks

TL;DR

Membership inference attacks reveal training data by exploiting distribution gaps between member and non-member predictions. Center-Based Relaxed Learning (CRL) is a training paradigm that combines RelaxLoss and Center Loss with logit normalization to minimize this gap while preserving (or even improving) generalization, making it architecture-agnostic. It introduces ImpRelaxLoss and Relaxed Center Loss to adaptively relax fitting on easy samples and tighten center-aligned representations, balancing privacy and accuracy. Extensive experiments across CIFAR-10/100, SVHN, and ArXiv-10 show CRL outperforming state-of-the-art defenses in privacy protection with comparable utility, suggesting practical, scalable privacy-preserving learning for diverse classifiers.

Abstract

Membership inference attacks (MIAs) are currently considered one of the main privacy attack strategies, and their defense mechanisms have also been extensively explored. However, there is still a gap between the existing defense approaches and ideal models in performance and deployment costs. In particular, we observed that the privacy vulnerability of the model is closely correlated with the gap between the model's data-memorizing ability and generalization ability. To address this, we propose a new architecture-agnostic training paradigm called center-based relaxed learning (CRL), which is adaptive to any classification model and provides privacy preservation by sacrificing a minimal or no loss of model generalizability. We emphasize that CRL can better maintain the model's consistency between member and non-member data. Through extensive experiments on standard classification datasets, we empirically show that this approach exhibits comparable performance without requiring additional model capacity or data costs.

Figures (8)

• Figure 1: Relationship between the distance to the origin and the distance to the decision boundary in Cross entropy, RelaxLoss, and our proposed approach, CRL. The blue points are Member data while the green points are non-member data. We compute the distance to the decision boundary by subtracting 2nd largest confidence from the 1st largest confidence. Therefore, more overlap between Member and Non-member is better for privacy.
• Figure 2: An overview of the proposed relaxed center loss function. The mini-batch determines which scenario to execute.
• Figure 3: An overview of CRL's training and testing phases. The parameters of centers are a part of the loss function but not a part of the model.
• Figure 4: Performance of defenses against adaptive attacks (ResNet18, CIFAR-10).
• Figure 5: Performance of defenses against adaptive attacks (ResNet18, CIFAR-100, data augmentation applied).