Rethinking On-Device LLM Reasoning: Why Analogical Mapping Outperforms Abstract Thinking for IoT DDoS Detection
William Pan, Guiran Liu, Binrong Zhu, Qun Wang, Yingzhou Lu, Beiyu Lin, Rose Qingyang Hu
TL;DR
The paper tackles edge-friendly IoT DDoS detection by placing ODLLMs at the network boundary and addressing their limited reasoning with a hybrid CoT–RAG framework. A teacher model generates CoT outputs to build a domain KB offline, while online inference uses a Prob-RAG approach that grounds small ODLLMs with exemplars and few-shot prompts, leveraging $x\in\mathbb{R}^9$ features and a class-probability vector $\mathbf p\in\Delta^{C}$ to retrieve prototypes $R_k(\mathbf p)$. Empirical results show that pure CoT underperforms and can hallucinate, whereas Few-Shot RAG with exemplar prompts significantly boosts macro-F1 up to around $0.85$ on diverse DDoS types; model scale further boosts performance, with Llama 3.2 3B achieving top results in Few-Shot settings. The work demonstrates that for structured, resource-constrained edge data, analogical mapping via exemplars is far more effective than abstract reasoning, enabling robust, privacy-preserving real-time threat detection at the edge.
Abstract
The rapid expansion of IoT deployments has intensified cybersecurity threats, notably Distributed Denial of Service (DDoS) attacks, characterized by increasingly sophisticated patterns. Leveraging Generative AI through On-Device Large Language Models (ODLLMs) provides a viable solution for real-time threat detection at the network edge, though limited computational resources present challenges for smaller ODLLMs. This paper introduces a novel detection framework that integrates Chain-of-Thought (CoT) reasoning with Retrieval-Augmented Generation (RAG), tailored specifically for IoT edge environments. We systematically evaluate compact ODLLMs, including LLaMA 3.2 (1B, 3B) and Gemma 3 (1B, 4B), using structured prompting and exemplar-driven reasoning strategies. Experimental results demonstrate substantial performance improvements with few-shot prompting, achieving macro-average F1 scores as high as 0.85. Our findings highlight the significant advantages of incorporating exemplar-based reasoning, underscoring that CoT and RAG approaches markedly enhance small ODLLMs' capabilities in accurately classifying complex network attacks under stringent resource constraints.