Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs

Yash Sinha, Murari Mandal, Mohan Kankanhalli

TL;DR

This paper proposes a novel concept of anti-sample-induced unlearning, and enables fine-grained targeted unlearning, allowing for the selective removal of specific associations without impacting related knowledge - something not achievable by previous works.

Abstract

The key components of machine learning are data samples for training, model for learning patterns, and loss function for optimizing accuracy. Analogously, unlearning can potentially be achieved through anti-data samples (or anti-samples), unlearning method, and reversed loss function. While prior research has explored unlearning methods and reversed loss functions, the potential of anti-samples remains largely untapped. In this paper, we introduce UnSTAR: Unlearning with Self-Taught Anti-Sample Reasoning for large language models (LLMs). Our contributions are threefold; first, we propose a novel concept of anti-sample-induced unlearning; second, we generate anti-samples by leveraging misleading rationales, which help reverse learned associations and accelerate the unlearning process; and third, we enable fine-grained targeted unlearning, allowing for the selective removal of specific associations without impacting related knowledge - something not achievable by previous works. Results demonstrate that anti-samples offer an efficient, targeted unlearning strategy for LLMs, opening new avenues for privacy-preserving machine learning and model modification.

UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs

TL;DR

This paper proposes a novel concept of anti-sample-induced unlearning, and enables fine-grained targeted unlearning, allowing for the selective removal of specific associations without impacting related knowledge - something not achievable by previous works.

Abstract

The key components of machine learning are data samples for training, model for learning patterns, and loss function for optimizing accuracy. Analogously, unlearning can potentially be achieved through anti-data samples (or anti-samples), unlearning method, and reversed loss function. While prior research has explored unlearning methods and reversed loss functions, the potential of anti-samples remains largely untapped. In this paper, we introduce UnSTAR: Unlearning with Self-Taught Anti-Sample Reasoning for large language models (LLMs). Our contributions are threefold; first, we propose a novel concept of anti-sample-induced unlearning; second, we generate anti-samples by leveraging misleading rationales, which help reverse learned associations and accelerate the unlearning process; and third, we enable fine-grained targeted unlearning, allowing for the selective removal of specific associations without impacting related knowledge - something not achievable by previous works. Results demonstrate that anti-samples offer an efficient, targeted unlearning strategy for LLMs, opening new avenues for privacy-preserving machine learning and model modification.

Paper Structure

This paper contains 14 sections, 2 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. UnStar
  4. Experiments and Results
  5. Experiments
  6. Results.
  7. Conclusion
  8. Appendix
  9. Used prompts
  10. Paraphrased questions
  11. Rephrased Answers.
  12. Generated misleading explanations.
  13. Answers generated after unlearning.
  14. Prompts used for evaluation

Figures (3)

  • Figure 1: An overview of UnStar. For a question-answer pair in the forget set, paraphrased questions and incorrect answers are generated using LLM. The justification is achieved through "rationalization" based on STaR. Following the unlearning of a question, more challenging paraphrased versions are generated to further enhance the unlearning process.
  • Figure 2: Performance of each criterion (normalized by maximum) on WPU dataset. Higher is better for all metrics. UnStar offers a balanced solution, enhancing unlearning efficacy and model utility while maintaining competitive performance in response quality, hallucination avoidance, and adversarial robustness.
  • Figure 3: Iterations vs. Unlearning Efficacy: As the LLM progressively unlearns multiple paraphrased versions of a question, its ability to accurately respond to correct answer decreases.