Large Language Model Assisted Adversarial Robustness Neural Architecture Search

TL;DR

This work addresses the challenge of adversarial robustness neural architecture search (ARNAS) by introducing LLMO, a large language model–guided optimizer that uses the CRISPE prompt framework to iteratively search for robust architectures within the NAS-Bench-201 space. By encoding candidate architectures as edge-bit arrays and treating Gemini’s outputs as solutions, LLMO leverages prompt refinement rather than traditional evolutionary operators, achieving competitive results against six meta-heuristics on CIFAR-10 and CIFAR-100 under various attacks. The study demonstrates that LLMs can serve as effective combinatorial optimizers for ARNAS, offering a user-friendly, transfer-function–free approach with potential applicability to other combinatorial domains. Overall, the findings highlight the promising role of LLM-driven optimization in designing robust neural architectures and suggest avenues for broader application and prompt-based methodology improvements.

Abstract

Motivated by the potential of large language models (LLMs) as optimizers for solving combinatorial optimization problems, this paper proposes a novel LLM-assisted optimizer (LLMO) to address adversarial robustness neural architecture search (ARNAS), a specific application of combinatorial optimization. We design the prompt using the standard CRISPE framework (i.e., Capacity and Role, Insight, Statement, Personality, and Experiment). In this study, we employ Gemini, a powerful LLM developed by Google. We iteratively refine the prompt, and the responses from Gemini are adapted as solutions to ARNAS instances. Numerical experiments are conducted on NAS-Bench-201-based ARNAS tasks with CIFAR-10 and CIFAR-100 datasets. Six well-known meta-heuristic algorithms (MHAs) including genetic algorithm (GA), particle swarm optimization (PSO), differential evolution (DE), and its variants serve as baselines. The experimental results confirm the competitiveness of the proposed LLMO and highlight the potential of LLMs as effective combinatorial optimizers. The source code of this research can be downloaded from \url{https://github.com/RuiZhong961230/LLMO}.

Figures (5)

• Figure 1: Components in the CRISPE framework.
• Figure 2: The architecture of NAS-Bench-201-based ARNAS search space.
• Figure 3: A demonstration of LLMO.
• Figure 4: Convergence curves of optimizers for ARNAS on CIFAR-10.
• Figure 5: Convergence curves of optimizers for ARNAS on CIFAR-100.