Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Global Challenge for Safe and Secure LLMs Track 1

Xiaojun Jia, Yihao Huang, Yang Liu, Peng Yan Tan, Weng Kuan Yau, Mun-Thye Mak, Xin Ming Sim, Wee Siong Ng, See Kiong Ng, Hanqing Liu, Lifeng Zhou, Huanqian Yan, Xiaobing Sun, Wei Liu, Long Wang, Yiming Qian, Yong Liu, Junxiao Yang, Zhexin Zhang, Leqi Lei, Renmiao Chen, Yida Lu, Shiyao Cui, Zizhou Wang, Shaohua Li, Yan Wang, Rick Siow Mong Goh, Liangli Zhen, Yingjie Zhang, Zhe Zhao

Abstract

This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks. With the increasing integration of LLMs in critical sectors such as healthcare, finance, and public administration, ensuring these models are resilient to adversarial attacks is vital for preventing misuse and upholding ethical standards. This competition focused on two distinct tracks designed to evaluate and enhance the robustness of LLM security frameworks. Track 1 tasked participants with developing automated methods to probe LLM vulnerabilities by eliciting undesirable responses, effectively testing the limits of existing safety protocols within LLMs. Participants were challenged to devise techniques that could bypass content safeguards across a diverse array of scenarios, from offensive language to misinformation and illegal activities. Through this process, Track 1 aimed to deepen the understanding of LLM vulnerabilities and provide insights for creating more resilient models.

Global Challenge for Safe and Secure LLMs Track 1

Abstract

This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks. With the increasing integration of LLMs in critical sectors such as healthcare, finance, and public administration, ensuring these models are resilient to adversarial attacks is vital for preventing misuse and upholding ethical standards. This competition focused on two distinct tracks designed to evaluate and enhance the robustness of LLM security frameworks. Track 1 tasked participants with developing automated methods to probe LLM vulnerabilities by eliciting undesirable responses, effectively testing the limits of existing safety protocols within LLMs. Participants were challenged to devise techniques that could bypass content safeguards across a diverse array of scenarios, from offensive language to misinformation and illegal activities. Through this process, Track 1 aimed to deepen the understanding of LLM vulnerabilities and provide insights for creating more resilient models.

Paper Structure

This paper contains 61 sections, 10 equations, 8 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Challenge Overview
  3. Objective.
  4. Duration
  5. Structure
  6. Definitions.
  7. Submissions and Deliverables
  8. Evaluation Metrics
  9. Track 1 Leaderboard Ranking
  10. Top 5 Team Methodologies and Approaches
  11. Method of Team DeepAttack
  12. Related Work
  13. Methodology
  14. Preliminaries
  15. The proposed SI-GCG attack method
  16. ...and 46 more sections

Figures (8)

  • Figure 1: An illustration of how the fixed harmful template and the optimized suffix guide an LLM to generate a harmful response.
  • Figure 2: The illustration of the proposed automatic optimal suffix selection strategy.
  • Figure 3: Example of the system prompt and crafted prompts. The adversarial behavior is inserted as goal in a prompt. An adversarial suffix is appended to a prompt.
  • Figure 4: Workflow of the jailbreak approach. Judge LLM can be either gemma-2-9b-instruct or GPT-4o in the experiments.
  • Figure 5: Combination attack. A mutation, a candidate template, and a candidate suffix are selected from the respective pool in each iteration.
  • ...and 3 more figures