TrojanGYM: A Detector-in-the-Loop LLM for Adaptive RTL Hardware Trojan Insertion

TL;DR

TrojanGYM tackles the problem of detector overfitting to narrow HT templates by introducing a detector-aware, agentic loop that co-evolves HT insertions and GNN-based detectors at the RTL level. It leverages multiple LLMs to analyze, plan, and synthesize HTs, while a Robust-GNN4TJ detector provides multi-view feedback that drives iterative refinement of HTs and properties. The approach yields diverse, functionally correct HTs and reveals robustness gaps in current detectors, achieving substantial evasion (up to 83.33% under oracle selection) and exposing detector blind spots not evident from TrustHub benchmarks. The work provides a framework, dataset, and evaluation methodology with practical implications for pre-silicon security research and underscores the need for detector-adaptive benchmarking in hardware security.

Abstract

Hardware Trojans (HTs) remain a critical threat because learning-based detectors often overfit to narrow trigger/payload patterns and small, stylized benchmarks. We introduce TrojanGYM, an agentic, LLM-driven framework that automatically curates HT insertions to expose detector blind spots while preserving design correctness. Given high-level HT specifications, a suite of cooperating LLM agents (instantiated with GPT-4, LLaMA-3.3-70B, and Gemini-2.5Pro) proposes and refines RTL modifications that realize diverse triggers and payloads without impacting normal functionality. TrojanGYM implements a feedback-driven benchmark generation loop co-designed with HT detectors, in which constraint-aware syntactic checking and GNN-based HT detectors provide feedback that iteratively refines HT specifications and insertion strategies to better surface detector blind spots. We further propose Robust-GNN4TJ, a new implementation of the GNN4TJ with improved graph extraction, training robustness, and prediction reliability, especially on LLM-generated HT designs. On the most challenging TrojanGYM-generated benchmarks, Robust-GNN4TJ raises HT detection rates from 0% to 60% relative to a prior GNN-based detector. We instantiate TrojanGYM on SRAM, AES-128, and UART designs at RTL level, and show that it systematically produces diverse, functionally correct HTs that reach up to 83.33% evasion rates against modern GNN-based detectors, revealing robustness gaps that are not apparent when these detectors are evaluated solely on existing TrustHub-style benchmarks. Post peer-review, we will release all codes and artifacts.

Figures (6)

• Figure 1: The HT insertion framework.
• Figure 2: t-SNE visualization of graph embeddings learned by Robust-GNN4TJ on the full training dataset. Clean designs are shown in green, and Trojan-inserted designs across all HT types are shown in red, with noticeable overlap indicating structural similarity between benign and Trojan designs.
• Figure 3: t-SNE visualization of graph embeddings learned by Robust-GNN4TJ models trained separately on different HT types. Clean designs are shown in green, while Trojan designs are color-coded by HT category. The clearer clustering across HT-specific embeddings suggests improved separability compared to joint training.
• Figure 4: Detection and evasion performance of Robust-GNN4TJ under iterative, detector-aware Trojan insertion. (a) Trojan detection accuracy decreases as LLM-driven insertion attempts progress, while (b) Trojan evasion success increases across iterations. Results are shown for different LLMs, with an optimal reference illustrating the upper bound of evasion performance.
• Figure 5: Edge-level structural similarity heatmaps between the original RTL design (ORI) and iteratively generated variants (A1–A4) across different LLMs and benchmarks. Each subplot corresponds to one (LLM, design) pair and visualizes pairwise edge Jaccard similarity over DFGs, highlighting how structural changes accumulate across successive LLM-driven transformations. Missing variants (e.g., absent A4) are marked as NA.